-- Logs begin at Thu 2022-10-20 14:01:41 UTC, end at Thu 2022-10-20 14:57:23 UTC. -- Oct 20 14:27:23.236661 dv-cinder-779233-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 20 14:27:23.245453 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 20 14:27:23.247460 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 20 14:27:23 2022] *** Oct 20 14:27:23.247460 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 20 14:27:23.247460 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 20 14:27:23.247460 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: nodename: dv-cinder-779233-7-iscsi Oct 20 14:27:23.247460 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: machine: x86_64 Oct 20 14:27:23.247460 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: clock source: unix Oct 20 14:27:23.247460 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: pcre jit disabled Oct 20 14:27:23.247460 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: detected number of CPU cores: 4 Oct 20 14:27:23.247460 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: current working directory: / Oct 20 14:27:23.248053 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: detected binary path: /usr/bin/uwsgi-core Oct 20 14:27:23.248053 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: your processes number limit is 47680 Oct 20 14:27:23.248053 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: your memory page size is 4096 bytes Oct 20 14:27:23.248053 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: detected max file descriptor number: 2048 Oct 20 14:27:23.248053 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: lock engine: pthread robust mutexes Oct 20 14:27:23.248053 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: thunder lock: enabled Oct 20 14:27:23.248053 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 20 14:27:23.248053 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 20 14:27:23.287031 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: Python main interpreter initialized at 0x55fd6a1bf9c0 Oct 20 14:27:23.287031 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: python threads support enabled Oct 20 14:27:23.287031 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: your server socket listen backlog is limited to 100 connections Oct 20 14:27:23.287031 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: your mercy for graceful operations on workers is 90 seconds Oct 20 14:27:23.287352 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: mapped 403077 bytes (393 KB) for 2 cores Oct 20 14:27:23.287445 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: *** Operational MODE: preforking *** Oct 20 14:27:23.287702 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: *** uWSGI is running in multiple interpreter mode *** Oct 20 14:27:23.287702 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: spawned uWSGI master process (pid: 82073) Oct 20 14:27:23.287894 dv-cinder-779233-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 20 14:27:23.292377 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: spawned uWSGI worker 1 (pid: 82084, cores: 1) Oct 20 14:27:23.292377 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: spawned uWSGI worker 2 (pid: 82085, cores: 1) Oct 20 14:27:23.292377 dv-cinder-779233-7-iscsi devstack@keystone.service[82073]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 20 14:27:27.547854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 14:27:27.547854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: from cryptography.utils import int_from_bytes Oct 20 14:27:27.548671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 14:27:27.548671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: from cryptography.utils import int_from_bytes Oct 20 14:27:27.549007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 14:27:27.549007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: from cryptography.utils import int_from_bytes Oct 20 14:27:27.549447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 14:27:27.549447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: from cryptography.utils import int_from_bytes Oct 20 14:27:27.598080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 14:27:27.598255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 14:27:27.598446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] command line args: [] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 14:27:27.598711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 14:27:27.598889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ================================================================================ {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 14:27:27.599031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 14:27:27.599143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] admin_token = **** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.599232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 14:27:27.599392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] config_dir = [] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.599476 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] command line args: [] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 14:27:27.599579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.599662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 14:27:27.599759 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] config_source = [] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.599837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ================================================================================ {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 14:27:27.599938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] debug = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.600081 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] admin_token = **** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.600157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.600479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] config_dir = [] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.600559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.600620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.600675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.600736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] config_source = [] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.600891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] debug = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] list_limit = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] log_config_append = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601821 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.601921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] log_dir = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] log_file = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] list_limit = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] log_config_append = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] log_dir = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.602941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] log_file = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603214 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603341 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.603957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.604075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.604211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.604313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.604475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.604596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.604707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.604843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.604953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.605058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.605208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.605323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.605457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.605578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.605684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.605827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.605936 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.606080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.606191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] publish_errors = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.606295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.606431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.606551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.607190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.607343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.607461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.607608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.607691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] publish_errors = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.607843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] standard_threads = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.607966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.608060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.608239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.608327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.608419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.608498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.608629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] use_journal = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.608713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.608789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] use_json = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.608906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.609021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] use_stderr = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.609123 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] use_syslog = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.609204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] standard_threads = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.609354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.609433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.609609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.609691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.609787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.609861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.609967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] use_journal = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.610108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.610204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] use_json = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.610314 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] use_stderr = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.610393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.610468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] use_syslog = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.610567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.610695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 14:27:27.610804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.610955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.611051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.611200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.611284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.611464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.611546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.611709 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.611785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.611962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.612046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.612205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.612281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.612468 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.612555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.612705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.612789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.612948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.613058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.613192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.613268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.613426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.613516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.613667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.613743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.613918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.613993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.614158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.614243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.614392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.614473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.614643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.614718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.614882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.615034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.615140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.615215 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.615390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.615467 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.615619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.615696 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.615848 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.615930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.616078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.616159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.616311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.616407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.616579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.616654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.616825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.616900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.617296 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.617381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.617533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.617608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.617776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.617850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.618016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.618098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.618248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.618322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.618493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.618568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.618722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.618799 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.618954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.619029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.619187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.619261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.619418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.619500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.619647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.619723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.619875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.619951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.620104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.620180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.620337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.620430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.620605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.620680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.620826 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.620899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.621059 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.621140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.621285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.621364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.621517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.621593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.621781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.621862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.621996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.622073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.622226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.622920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.623058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.623177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.623244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.623338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.623457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.623582 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.623691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.623824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.623925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.624077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.624143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.624418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.624544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.624724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.624841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.624991 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.625105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.625240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.625346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.625459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.625597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.625703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.625816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.625960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.626065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.626170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.626303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.626416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.626528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.626676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.626783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.626888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.627031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.627137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.627241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.627388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.627496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.627630 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.627738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.627843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.628049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.628170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.628297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.628297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.628629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.628868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.629036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.629269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.629394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.629563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.629685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.629809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.629990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.630135 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.630268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.630402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.630531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.630706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.630706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.630954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.631085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.631214 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.631355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.631486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.631616 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.631758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.631887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.632068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.632206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.632335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.632550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.632693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.632833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.632965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.633103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.633234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.633361 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.633496 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.633620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.633734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.633852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.633973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.634098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.634286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.634365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.634433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.634508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.634613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.634737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.634870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635010 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635403 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635663 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635727 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.635995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] auth.external = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636569 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636882 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] auth.password = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.636882 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] auth.token = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] auth.external = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.637884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.638052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.638201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.638272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] credential.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.638441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.638509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.638679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] auth.password = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.638781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.638913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] auth.token = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.638992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.639162 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.639244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.639404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.639486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.639656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.639732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.639899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.640149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.640226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.640420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.640505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.640656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.640737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.640899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.640976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.641143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] credential.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.641223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.641394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.641472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.641647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.641724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.641898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.641977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.642131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] federation.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.642214 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.642367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.642445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.642604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.642682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.642847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.642931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.643076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.643154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.643323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.643403 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.643559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.643636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.643810 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.643895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.644044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.644122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.644295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.644386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.644558 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.644647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.644804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.644881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.645040 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.645118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] federation.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.645292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.645377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.645539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.645616 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.645774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.645851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.646008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.646093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.646253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.646333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.646483 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.646567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.646717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.646873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.646948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.647016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.647201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.647277 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.647430 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.647517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.647659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.647737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.647886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.647963 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.648128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.648214 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.648391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.648473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.648633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.648709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.648866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.648964 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.649114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.649193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.649348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.649425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.649591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.649683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.649826 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.649903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.650054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.650132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.650287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.650374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.650514 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.650592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.650745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.650821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.650979 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.651064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.651216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.651294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.651469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.651585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.651729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.651815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.651981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.652059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.652255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.652332 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.652520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.652610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.652763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.652842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.652998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.653075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.653228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.653312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.653458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.653551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.653713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.653788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.653949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.654019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.654202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.654354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.654574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.654652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.654813 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.654896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.655124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.655212 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.655303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.655381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.655541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.655619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.655770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.655847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.656000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.656076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.656247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.656325 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.656509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.656587 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.656751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.656835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.656984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.657065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.657226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.657304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.657472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.657551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.657706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.657785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.657945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.658022 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.658182 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.658259 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.658429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.658509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.658662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.658738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.658895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.658973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.659127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.659205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.659355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.659432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.659619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.659705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.659821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.659900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.660056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.660133 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.660283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.660380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.660551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.660630 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.660794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.660870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.661035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.661111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.661278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.661356 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.661520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.661597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.661748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.661825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.661991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.662070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.662221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.662296 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.662453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.662529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.662687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.662764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.662918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.662994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.663149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.663224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.663391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.663469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.663624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.663702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.663873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.663949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.664098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.664175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.664390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.664472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.664592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.664669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.664827 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.664907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.665060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.665138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.665299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.665376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.665537 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.665614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.665764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.665841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.665995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.666073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.666232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] resource.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.666311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.666478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.666554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.666721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.666797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.666954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.667033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.667201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.667277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.667443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.667521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.667677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.667755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.667916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.667992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.668149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.668225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.668408 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.668491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.668646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] role.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.668723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.668883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.668970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] role.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.669111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.669731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] resource.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.669888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.669956 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.670182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.670353 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.670432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.670502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.670615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.670694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.670869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.670955 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.671108 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.671186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.671358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.671435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.671597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.671684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.671825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.671903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.672063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] role.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.672140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.672305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.672419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] role.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.672563 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.672640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.672806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.672883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.673056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.673200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.673307 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.673434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.673558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.673683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.673807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.673931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.674056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.674178 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.674304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.674422 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.674554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.674641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.674788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.674893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.675039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.675144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.675289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.675393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.675540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.675618 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.675790 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.675868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.676036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.676191 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.676269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.676457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.676538 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.676715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.676805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.676971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.677056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.677224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.677308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.677450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.677535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.677686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] token.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.677764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.677922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.677999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.678168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.678254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.678405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.678482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.678639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.678715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.678873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.678960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.679107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.679185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.679346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.679422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.679587 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.679673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.679831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.679916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.680065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.680142 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.680300 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.680434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.680558 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.680636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.680800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] token.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.680876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.681045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.681131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.681284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.681370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.681524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.681603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.681782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.681858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.682032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.682118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.connection = **** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.682272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.682349 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.682515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.682592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.682749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.682827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.682988 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.683066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.683223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.683300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.683458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.683537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.683693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.683777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.683926 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.684004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.684167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.684247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.684425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.684507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.684670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.684782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.685526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.685715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.685788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.685849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.connection = **** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.686002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.686127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.686245 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.686329 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.686473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.686555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.686709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.686788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.686943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.687028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.687111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82085) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 14:27:27.687246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.687487 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.687728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.687977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.688217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.688489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.688731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.688982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.689221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.689477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.689714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.689960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.690202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 14:27:27.690375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 14:27:27.715883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.716168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.716623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.719033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.719270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.719503 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.724674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.727812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.753896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.754127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.755144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.755380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.806747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.807773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.855228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.855399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.856568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:27.856870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.011966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.012196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.012421 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.012512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.012655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.012768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.012876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.012957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.013120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.013192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.013341 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.013552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.036607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.036846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.037104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.037334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.037474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.037609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.037742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.037885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.038016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.038155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.038284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.038284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.038528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.038657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.050119 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.051026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.062794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.063032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.063287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.063432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.063560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.063684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.063811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.063937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.064061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.064194 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.064194 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.064453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.064588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.064588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82085) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 14:27:28.080988 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.081132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.081217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 14:27:28.081334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 14:27:28.081525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.081682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.081778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 14:27:28.081883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 14:27:28.081990 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.082159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.082240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.082427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.082509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.082691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.082771 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.082951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.092832 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.093107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.093336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.093424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.093618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.093701 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.093890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.094175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.100536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.101154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.103399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.103685 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.103969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.104049 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.104252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.104333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.104560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.104655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.104823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.104904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.105092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.105177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.105405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.105494 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.105635 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.105715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.105907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.105990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.106173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.106253 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.106447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.107319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.130170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.130807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.131641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.131929 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.132260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.132571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.136613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.136902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.137190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.137270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.137475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.137752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.143520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.143812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.144292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.144601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.149765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.150069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.150192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.150362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.150486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.150648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.150771 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.150925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.151056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.151203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.151335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.151476 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.151613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.151747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.151883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.152149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.168065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.168232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.168359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.168535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.172279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.172393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.175500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.175583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.175797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.175879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.179959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.180057 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.183038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.183121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.186846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.186942 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.187148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.187232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.187439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.187521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.187721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.187810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.188003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.188083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.188289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.188388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.203861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.204153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.204233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.204450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.204535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.204723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.204817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.205010 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.211556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.211762 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.213117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.213311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.214641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.214831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.214912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.215114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.217671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.217777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.217959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.218069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.218250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.218352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.218535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.218637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.218942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.219031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.230033 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.230138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.230314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.230396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.230592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.230681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.230870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.230952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.231251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.231331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.231540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.232169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.232400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.232523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.247913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.248395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.250800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.251303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.253663 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.254198 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.255111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.255704 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.256576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.256861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.257262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.257348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.257585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.257990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.264071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.264906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.265555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.266375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.268432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.268831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.269115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.269268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.269390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.269665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.269954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.270236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.277089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.277380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.277660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.278030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.278324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.278671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.284685 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82085) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.284974 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.285254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.285538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 14:27:28.285620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.285808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.285894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.286078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.286170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.286350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.286431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.286622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.286699 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.286899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.286993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.287172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.287254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.287451 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.287532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.287732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.287812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.288005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.288129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.288294 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82085) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.288391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.288583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.288858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.289127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 14:27:28.322308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fd6a1bf9c0 pid: 82085 (default app) Oct 20 14:27:28.323407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fd6a1bf9c0 pid: 82084 (default app) Oct 20 14:27:28.326075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252972a1-942d-4417-a376-228d9aaaf34e None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:28.326355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252972a1-942d-4417-a376-228d9aaaf34e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:28.326627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252972a1-942d-4417-a376-228d9aaaf34e None None] PATH_INFO: `/v3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:28.330745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Thu Oct 20 14:27:28 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:35.108804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5a7d4-1eab-4b00-bbc2-e60655929235 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:35.109330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5a7d4-1eab-4b00-bbc2-e60655929235 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:35.109330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5a7d4-1eab-4b00-bbc2-e60655929235 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:35.114050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:27:35.119705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:35.120275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:35.120275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:35.192863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: INFO dbcounter [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] Registered counter for database keystone Oct 20 14:27:35.212109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG oslo_db.sqlalchemy.engines [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82085) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 14:27:35.218918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writer thread running {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 14:27:35.395725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG passlib.handlers.bcrypt [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=82085) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 20 14:27:35.396044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG passlib.handlers.bcrypt [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82085) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 20 14:27:35.453413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:35.454992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:35.581573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 20 14:27:36.010479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5a7d4-1eab-4b00-bbc2-e60655929235 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:36.081806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: INFO dbcounter [None req-0dc5a7d4-1eab-4b00-bbc2-e60655929235 None None] Registered counter for database keystone Oct 20 14:27:36.102717 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG oslo_db.sqlalchemy.engines [None req-0dc5a7d4-1eab-4b00-bbc2-e60655929235 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82084) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 14:27:36.109174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writer thread running {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 14:27:36.227023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5a7d4-1eab-4b00-bbc2-e60655929235 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:36.231231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c36510-7076-4a53-9b22-17b8ee2b49f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:36.231958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c36510-7076-4a53-9b22-17b8ee2b49f6 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:36.232233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c36510-7076-4a53-9b22-17b8ee2b49f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:36.232530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c36510-7076-4a53-9b22-17b8ee2b49f6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:36.240565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c36510-7076-4a53-9b22-17b8ee2b49f6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:36.293461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f6c36510-7076-4a53-9b22-17b8ee2b49f6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 20 14:27:36.293762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c36510-7076-4a53-9b22-17b8ee2b49f6 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:36.303196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Thu Oct 20 14:27:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 14:27:36.309108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:36.327617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e811f-581e-4f81-b32c-d0a371e9cdd8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:36.332066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ec9f1-4cbf-4a61-a47b-38da7e28e3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:36.333189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ec9f1-4cbf-4a61-a47b-38da7e28e3fd admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:36.333468 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ec9f1-4cbf-4a61-a47b-38da7e28e3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:36.333737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ec9f1-4cbf-4a61-a47b-38da7e28e3fd admin admin] PATH_INFO: `/v3/projects/cfa845f715934496950bc728f50bbff3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:36.335317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ec9f1-4cbf-4a61-a47b-38da7e28e3fd admin admin] RBAC: Authorizing `identity:get_project(project_id=cfa845f715934496950bc728f50bbff3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:36.390999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6b6ec9f1-4cbf-4a61-a47b-38da7e28e3fd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 20 14:27:36.391356 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ec9f1-4cbf-4a61-a47b-38da7e28e3fd admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:36.395962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Thu Oct 20 14:27:36 2022] GET /identity/v3/projects/cfa845f715934496950bc728f50bbff3 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:38.488627 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0003174-1842-4d67-a6e6-26ee821e28ff None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:38.489135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0003174-1842-4d67-a6e6-26ee821e28ff None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:38.489217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0003174-1842-4d67-a6e6-26ee821e28ff None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:38.493472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:27:38.498874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87369fca-1c1e-4f76-927b-f4c5e3330d27 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:38.499247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87369fca-1c1e-4f76-927b-f4c5e3330d27 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:38.499434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87369fca-1c1e-4f76-927b-f4c5e3330d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:38.519598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-87369fca-1c1e-4f76-927b-f4c5e3330d27 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:38.521139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-87369fca-1c1e-4f76-927b-f4c5e3330d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:38.529522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 14:27:38.793343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0003174-1842-4d67-a6e6-26ee821e28ff None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:38.805310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0003174-1842-4d67-a6e6-26ee821e28ff None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:38.809477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19e35e-a5bc-41c6-aec8-d2d9e3c47b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:38.810600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19e35e-a5bc-41c6-aec8-d2d9e3c47b13 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:38.810872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19e35e-a5bc-41c6-aec8-d2d9e3c47b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:38.811139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19e35e-a5bc-41c6-aec8-d2d9e3c47b13 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:38.818525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19e35e-a5bc-41c6-aec8-d2d9e3c47b13 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:38.820267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19e35e-a5bc-41c6-aec8-d2d9e3c47b13 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:38.829006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Thu Oct 20 14:27:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:27:38.834424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87369fca-1c1e-4f76-927b-f4c5e3330d27 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:38.843788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87369fca-1c1e-4f76-927b-f4c5e3330d27 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:38.848033 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35fe70-cc24-4768-8820-d9fcb782ac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:38.848682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35fe70-cc24-4768-8820-d9fcb782ac9d admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:38.848961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35fe70-cc24-4768-8820-d9fcb782ac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:38.849230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35fe70-cc24-4768-8820-d9fcb782ac9d admin admin] PATH_INFO: `/v3/users/75ee7ae2d2194fbf9ae79851bb412a22` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:38.850516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35fe70-cc24-4768-8820-d9fcb782ac9d admin admin] RBAC: Authorizing `identity:get_user(user_id=75ee7ae2d2194fbf9ae79851bb412a22)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:38.852336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35fe70-cc24-4768-8820-d9fcb782ac9d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:38.853833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Thu Oct 20 14:27:38 2022] GET /identity/v3/users/75ee7ae2d2194fbf9ae79851bb412a22 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:27:41.042695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be445b-75b2-447d-882e-9dd19cc90956 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:41.043261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be445b-75b2-447d-882e-9dd19cc90956 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:41.043261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be445b-75b2-447d-882e-9dd19cc90956 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:41.045309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:41.050177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba751e8-ae49-438f-a425-48213b647ad3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:41.050601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba751e8-ae49-438f-a425-48213b647ad3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:41.050725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba751e8-ae49-438f-a425-48213b647ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:41.068250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-0ba751e8-ae49-438f-a425-48213b647ad3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:41.069449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0ba751e8-ae49-438f-a425-48213b647ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:41.094514 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18a557-59eb-4f4f-9cad-a033d04fdb05 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:41.094803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18a557-59eb-4f4f-9cad-a033d04fdb05 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:41.095079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18a557-59eb-4f4f-9cad-a033d04fdb05 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:41.096387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:27:41.101265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764bf9-a75d-4ac2-a0c0-56a37042f70b None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:41.101551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764bf9-a75d-4ac2-a0c0-56a37042f70b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:41.101831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764bf9-a75d-4ac2-a0c0-56a37042f70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:41.143443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG passlib.handlers.bcrypt [None req-2b764bf9-a75d-4ac2-a0c0-56a37042f70b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=82084) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 20 14:27:41.143847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG passlib.handlers.bcrypt [None req-2b764bf9-a75d-4ac2-a0c0-56a37042f70b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82084) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 20 14:27:41.150742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:27:41.193119 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-2b764bf9-a75d-4ac2-a0c0-56a37042f70b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:41.194685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2b764bf9-a75d-4ac2-a0c0-56a37042f70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:41.294903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:27:41.427537 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba751e8-ae49-438f-a425-48213b647ad3 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:41.452893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:41.532406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba751e8-ae49-438f-a425-48213b647ad3 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:41.557550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764bf9-a75d-4ac2-a0c0-56a37042f70b None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:41.566475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53665-ab1a-48a4-bdc9-f3244c30a498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:41.567116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53665-ab1a-48a4-bdc9-f3244c30a498 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:41.567399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53665-ab1a-48a4-bdc9-f3244c30a498 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:41.567679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53665-ab1a-48a4-bdc9-f3244c30a498 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:41.578999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb53665-ab1a-48a4-bdc9-f3244c30a498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-6cb53665-ab1a-48a4-bdc9-f3244c30a498 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:27:41.587897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:27:41.588872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:27:41.589893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:27:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:27:41.595024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53665-ab1a-48a4-bdc9-f3244c30a498 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:41.660422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764bf9-a75d-4ac2-a0c0-56a37042f70b None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:41.662638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:41.676719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:41.694648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53665-ab1a-48a4-bdc9-f3244c30a498 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:41.699080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:41.699942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:41.700761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:41.701036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:41.702002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:41.703415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:41.714140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.sql.core [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 14:27:41.714140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 14:27:41.714140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [parameters: {'id': '9d94015c533a4fae8f78b1f1c8b1a6af', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 14:27:41.714140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82084) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 14:27:41.728396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6e32d-98c1-4dbf-a84f-cf1d397828d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:41.729009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6e32d-98c1-4dbf-a84f-cf1d397828d8 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:41.729292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6e32d-98c1-4dbf-a84f-cf1d397828d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:41.729566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6e32d-98c1-4dbf-a84f-cf1d397828d8 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:41.730240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6e32d-98c1-4dbf-a84f-cf1d397828d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:27:41.730562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6e32d-98c1-4dbf-a84f-cf1d397828d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:41.732246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6e32d-98c1-4dbf-a84f-cf1d397828d8 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:27:41.733289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application result.read() Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 20 14:27:41.734125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application yield resource Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise self.value Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application session.commit() Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 14:27:41.735038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.session.flush() Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application compat.raise_( Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 14:27:41.735889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 14:27:41.736807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application result.read() Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:27:41.737571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [parameters: {'id': '9d94015c533a4fae8f78b1f1c8b1a6af', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 20 14:27:41.738358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:27:41.740788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 8/15] 10.222.0.13 () {60 vars in 1137 bytes} [Thu Oct 20 14:27:41 2022] POST /identity/v3/domains => generated 190 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:27:41.742103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:41.745717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 8/16] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:27:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:27:41.751661 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6e32d-98c1-4dbf-a84f-cf1d397828d8 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:41.839851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd792a9e-bdcf-4951-a138-ddcf6362f1d4 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:41.848745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6e32d-98c1-4dbf-a84f-cf1d397828d8 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:41.873928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4659d6-0dda-481e-a891-98a4ff60f9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:41.874773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4659d6-0dda-481e-a891-98a4ff60f9d3 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:41.875055 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4659d6-0dda-481e-a891-98a4ff60f9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:41.875328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4659d6-0dda-481e-a891-98a4ff60f9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:41.880961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b89e8e-940a-48a4-a398-eb7d6422deb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:41.881579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b89e8e-940a-48a4-a398-eb7d6422deb5 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:41.881856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b89e8e-940a-48a4-a398-eb7d6422deb5 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:41.882128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b89e8e-940a-48a4-a398-eb7d6422deb5 None admin] PATH_INFO: `/v3/users/75ee7ae2d2194fbf9ae79851bb412a22` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:41.883243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b89e8e-940a-48a4-a398-eb7d6422deb5 None admin] RBAC: Authorizing `identity:get_user(user_id=75ee7ae2d2194fbf9ae79851bb412a22)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:41.884716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b89e8e-940a-48a4-a398-eb7d6422deb5 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:41.886251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:27:41 2022] GET /identity/v3/users/75ee7ae2d2194fbf9ae79851bb412a22 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:27:41.892265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b89e8e-940a-48a4-a398-eb7d6422deb5 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:41.894088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4659d6-0dda-481e-a891-98a4ff60f9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:41.895616 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4659d6-0dda-481e-a891-98a4ff60f9d3 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:41.908117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-bd4659d6-0dda-481e-a891-98a4ff60f9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:27:41.912466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:27:41.913315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:27:41.914096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:27:41.914096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:27:41.914096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:41.914096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:27:41.920535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4659d6-0dda-481e-a891-98a4ff60f9d3 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:41.942913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:41.995874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b89e8e-940a-48a4-a398-eb7d6422deb5 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:42.020690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4659d6-0dda-481e-a891-98a4ff60f9d3 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:42.029548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69cafa-157c-4349-8310-014242921736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:42.030171 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69cafa-157c-4349-8310-014242921736 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:42.030564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69cafa-157c-4349-8310-014242921736 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:42.030848 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69cafa-157c-4349-8310-014242921736 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:42.032250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69cafa-157c-4349-8310-014242921736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:42.033779 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69cafa-157c-4349-8310-014242921736 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:42.035485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:27:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:27:42.040976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69cafa-157c-4349-8310-014242921736 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:42.054070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc67d4c-5f7f-4416-b348-1ea1acb8f5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:42.054674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc67d4c-5f7f-4416-b348-1ea1acb8f5e2 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:42.054953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc67d4c-5f7f-4416-b348-1ea1acb8f5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:42.055223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc67d4c-5f7f-4416-b348-1ea1acb8f5e2 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:42.055865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc67d4c-5f7f-4416-b348-1ea1acb8f5e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:27:42.056165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc67d4c-5f7f-4416-b348-1ea1acb8f5e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:42.057584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc67d4c-5f7f-4416-b348-1ea1acb8f5e2 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:42.077743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:27:42.126148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:42.146544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69cafa-157c-4349-8310-014242921736 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:42.183357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e45f25-f43f-4c90-8567-22d90af8f280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:42.184038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e45f25-f43f-4c90-8567-22d90af8f280 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:42.184318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e45f25-f43f-4c90-8567-22d90af8f280 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:42.185237 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e45f25-f43f-4c90-8567-22d90af8f280 None admin] PATH_INFO: `/v3/domains/default/users/75ee7ae2d2194fbf9ae79851bb412a22/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:42.187391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e45f25-f43f-4c90-8567-22d90af8f280 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=75ee7ae2d2194fbf9ae79851bb412a22, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:42.188947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e45f25-f43f-4c90-8567-22d90af8f280 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:42.303716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 11/21] 10.222.0.13 () {58 vars in 1361 bytes} [Thu Oct 20 14:27:42 2022] PUT /identity/v3/domains/default/users/75ee7ae2d2194fbf9ae79851bb412a22/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:27:44.417108 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296fb3-69da-474d-9cb3-009925dc30df None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:44.417602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296fb3-69da-474d-9cb3-009925dc30df None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:44.417672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296fb3-69da-474d-9cb3-009925dc30df None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:44.419470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:44.438556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f8b16-16a1-4fff-b36b-48f5ea3e2a3e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:44.439004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f8b16-16a1-4fff-b36b-48f5ea3e2a3e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:44.439206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f8b16-16a1-4fff-b36b-48f5ea3e2a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:44.469467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-746f8b16-16a1-4fff-b36b-48f5ea3e2a3e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:44.471067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-746f8b16-16a1-4fff-b36b-48f5ea3e2a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:44.561931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:27:44.785398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32677e-0fdc-42ce-9776-d167445bb5cb None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:44.785761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32677e-0fdc-42ce-9776-d167445bb5cb None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:44.786019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32677e-0fdc-42ce-9776-d167445bb5cb None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:44.787418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:44.806075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9710b67-d070-4c30-b5a5-fcfc08195ab8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:44.806372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9710b67-d070-4c30-b5a5-fcfc08195ab8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:44.806658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9710b67-d070-4c30-b5a5-fcfc08195ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:44.836168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d9710b67-d070-4c30-b5a5-fcfc08195ab8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:44.837507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d9710b67-d070-4c30-b5a5-fcfc08195ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:44.918030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:44.922774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:27:44.966761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32677e-0fdc-42ce-9776-d167445bb5cb None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:45.067826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:45.097359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32677e-0fdc-42ce-9776-d167445bb5cb None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:45.143296 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ab813-7f02-4414-865d-d95fe2263c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:45.143931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ab813-7f02-4414-865d-d95fe2263c9d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:45.144212 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ab813-7f02-4414-865d-d95fe2263c9d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:45.144538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ab813-7f02-4414-865d-d95fe2263c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:45.160378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ab813-7f02-4414-865d-d95fe2263c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:45.161942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ab813-7f02-4414-865d-d95fe2263c9d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-680ab813-7f02-4414-865d-d95fe2263c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:27:45.182763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:27:45.183678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:27:45.184531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:27:45.184531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:27:45.184531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:45.184531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 13/26] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:27:45.193556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9710b67-d070-4c30-b5a5-fcfc08195ab8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:45.317687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9710b67-d070-4c30-b5a5-fcfc08195ab8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:45.363925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ace952-bf50-4055-95c6-b98885d12869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:45.364630 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace952-bf50-4055-95c6-b98885d12869 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:45.364978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace952-bf50-4055-95c6-b98885d12869 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:45.365259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace952-bf50-4055-95c6-b98885d12869 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:45.365970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace952-bf50-4055-95c6-b98885d12869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:27:45.366282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace952-bf50-4055-95c6-b98885d12869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:45.368132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace952-bf50-4055-95c6-b98885d12869 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:45.373385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ab813-7f02-4414-865d-d95fe2263c9d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:45.400248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 14/27] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:45.405717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43644b-07b4-4b3f-8d77-cf282eee0247 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:45.405997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43644b-07b4-4b3f-8d77-cf282eee0247 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:45.406282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43644b-07b4-4b3f-8d77-cf282eee0247 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:45.407805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 15/28] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:45.410127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43644b-07b4-4b3f-8d77-cf282eee0247 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:45.485160 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:45.547806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ab813-7f02-4414-865d-d95fe2263c9d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:45.575229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43644b-07b4-4b3f-8d77-cf282eee0247 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:45.589736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:45.598920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b1c42-d49e-45ec-a133-d25ff045aebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:45.599542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b1c42-d49e-45ec-a133-d25ff045aebd None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:45.599820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b1c42-d49e-45ec-a133-d25ff045aebd None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:45.600096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b1c42-d49e-45ec-a133-d25ff045aebd None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:45.600812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b1c42-d49e-45ec-a133-d25ff045aebd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:45.602249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b1c42-d49e-45ec-a133-d25ff045aebd None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:45.654249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b48f1-1aef-4165-92ce-adea60ce1bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:45.655120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b48f1-1aef-4165-92ce-adea60ce1bbf None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:45.655413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b48f1-1aef-4165-92ce-adea60ce1bbf None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:45.655688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b48f1-1aef-4165-92ce-adea60ce1bbf None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:45.656379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b48f1-1aef-4165-92ce-adea60ce1bbf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:45.657944 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b48f1-1aef-4165-92ce-adea60ce1bbf None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:45.692418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 16/29] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:27:45 2022] POST /identity/v3/projects => generated 301 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:27:45.693862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae5d9d-ab74-4ec8-b7aa-42589c7dd444 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:45.694159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae5d9d-ab74-4ec8-b7aa-42589c7dd444 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:45.694440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae5d9d-ab74-4ec8-b7aa-42589c7dd444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:45.715305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-82ae5d9d-ab74-4ec8-b7aa-42589c7dd444 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:45.717236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-82ae5d9d-ab74-4ec8-b7aa-42589c7dd444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:45.755408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 14/30] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Oct 20 14:27:45 2022] POST /identity/v3/roles => generated 219 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:27:45.757064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a360f40-5a88-4b54-828c-838fd2a9f54c None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:45.757353 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a360f40-5a88-4b54-828c-838fd2a9f54c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:45.757630 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a360f40-5a88-4b54-828c-838fd2a9f54c None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:45.759509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 15/31] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:45.765743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1570f7-9e3a-427b-b10b-1f565c05a440 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:45.766286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1570f7-9e3a-427b-b10b-1f565c05a440 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:45.766631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1570f7-9e3a-427b-b10b-1f565c05a440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:45.805867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 17/32] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:27:45.817684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-5b1570f7-9e3a-427b-b10b-1f565c05a440 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:45.819267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5b1570f7-9e3a-427b-b10b-1f565c05a440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:45.919623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 16/33] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 20 14:27:46.119318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae5d9d-ab74-4ec8-b7aa-42589c7dd444 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:46.171824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:46.236772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae5d9d-ab74-4ec8-b7aa-42589c7dd444 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:46.279525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b1295-a63e-4d5c-8b95-9f70d64f5dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:46.280189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1295-a63e-4d5c-8b95-9f70d64f5dc6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.280504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1295-a63e-4d5c-8b95-9f70d64f5dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.280788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1295-a63e-4d5c-8b95-9f70d64f5dc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.289832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe88760-488a-4c90-b6ce-d139bd56f315 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.290123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe88760-488a-4c90-b6ce-d139bd56f315 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.290404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe88760-488a-4c90-b6ce-d139bd56f315 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.291733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 17/34] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:46.292053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b1295-a63e-4d5c-8b95-9f70d64f5dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-527b1295-a63e-4d5c-8b95-9f70d64f5dc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:27:46.297454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:27:46.298283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:27:46.299556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:27:46.300107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba5750-6c3f-45b3-843b-0a96729c28b8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.300107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba5750-6c3f-45b3-843b-0a96729c28b8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.300107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba5750-6c3f-45b3-843b-0a96729c28b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.300289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 18/35] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:27:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:27:46.303804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b1295-a63e-4d5c-8b95-9f70d64f5dc6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:46.326286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-25ba5750-6c3f-45b3-843b-0a96729c28b8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:46.327865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-25ba5750-6c3f-45b3-843b-0a96729c28b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:46.414807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 18/36] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:27:46.415989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba5750-6c3f-45b3-843b-0a96729c28b8 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:46.478718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:46.536122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b1295-a63e-4d5c-8b95-9f70d64f5dc6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:46.537333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba5750-6c3f-45b3-843b-0a96729c28b8 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:46.537764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:46.572408 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f331f6-e02e-4c0d-b5a4-80621c94af46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:46.572718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeef11-13ca-449d-9175-90b4aff2cfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:46.573138 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f331f6-e02e-4c0d-b5a4-80621c94af46 None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.573310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeef11-13ca-449d-9175-90b4aff2cfd1 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.573398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f331f6-e02e-4c0d-b5a4-80621c94af46 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.573555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeef11-13ca-449d-9175-90b4aff2cfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.573639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f331f6-e02e-4c0d-b5a4-80621c94af46 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.573824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeef11-13ca-449d-9175-90b4aff2cfd1 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.574257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f331f6-e02e-4c0d-b5a4-80621c94af46 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:46.574488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfeef11-13ca-449d-9175-90b4aff2cfd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:27:46.574798 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfeef11-13ca-449d-9175-90b4aff2cfd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:46.575934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f331f6-e02e-4c0d-b5a4-80621c94af46 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:46.576623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfeef11-13ca-449d-9175-90b4aff2cfd1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:46.596512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 19/37] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:27:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:46.596512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fd5c0-bab0-4b12-9054-004f8d1a108a None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.596512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fd5c0-bab0-4b12-9054-004f8d1a108a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.596512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fd5c0-bab0-4b12-9054-004f8d1a108a None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.596512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 20/38] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:46.598151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 19/39] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Oct 20 14:27:46 2022] POST /identity/v3/roles => generated 225 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:27:46.598434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fd5c0-bab0-4b12-9054-004f8d1a108a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:46.599555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d1692-2a7e-4954-94a1-7f2d63f28e95 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.599848 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d1692-2a7e-4954-94a1-7f2d63f28e95 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.600131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d1692-2a7e-4954-94a1-7f2d63f28e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.621131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ec4d1692-2a7e-4954-94a1-7f2d63f28e95 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:46.622484 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ec4d1692-2a7e-4954-94a1-7f2d63f28e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:46.702094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 20/40] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:27:46.703105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d1692-2a7e-4954-94a1-7f2d63f28e95 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:46.738130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fd5c0-bab0-4b12-9054-004f8d1a108a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:46.759102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:46.787707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a9f4f-c525-4a82-8d92-8a42479eb98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:46.788779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a9f4f-c525-4a82-8d92-8a42479eb98a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.788779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a9f4f-c525-4a82-8d92-8a42479eb98a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.788938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a9f4f-c525-4a82-8d92-8a42479eb98a None admin] PATH_INFO: `/v3/users/75ee7ae2d2194fbf9ae79851bb412a22` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.790356 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a9f4f-c525-4a82-8d92-8a42479eb98a None admin] RBAC: Authorizing `identity:get_user(user_id=75ee7ae2d2194fbf9ae79851bb412a22)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:46.791756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a9f4f-c525-4a82-8d92-8a42479eb98a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:46.793348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 21/41] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:27:46 2022] GET /identity/v3/users/75ee7ae2d2194fbf9ae79851bb412a22 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:46.798903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a9f4f-c525-4a82-8d92-8a42479eb98a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:46.818394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:46.831346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d1692-2a7e-4954-94a1-7f2d63f28e95 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:46.867001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88d5b6-3373-4cfd-a1d1-631bc2ec4c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:46.867882 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88d5b6-3373-4cfd-a1d1-631bc2ec4c71 None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.868236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88d5b6-3373-4cfd-a1d1-631bc2ec4c71 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.868621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88d5b6-3373-4cfd-a1d1-631bc2ec4c71 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.869371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88d5b6-3373-4cfd-a1d1-631bc2ec4c71 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:46.872242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88d5b6-3373-4cfd-a1d1-631bc2ec4c71 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:46.889534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 21/42] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Oct 20 14:27:46 2022] POST /identity/v3/roles => generated 223 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:27:46.929876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a9f4f-c525-4a82-8d92-8a42479eb98a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:46.968546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5737c-b37b-4913-90f7-913ed37ed74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:46.969207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5737c-b37b-4913-90f7-913ed37ed74e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:46.969555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5737c-b37b-4913-90f7-913ed37ed74e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:46.969836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5737c-b37b-4913-90f7-913ed37ed74e None admin] PATH_INFO: `/v3/domains/default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:46.971053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5737c-b37b-4913-90f7-913ed37ed74e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:46.972559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5737c-b37b-4913-90f7-913ed37ed74e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:46.974429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 22/43] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:27:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:27:46.980019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5737c-b37b-4913-90f7-913ed37ed74e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:46.980211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88d5b6-3373-4cfd-a1d1-631bc2ec4c71 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:47.040911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:47.098171 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88d5b6-3373-4cfd-a1d1-631bc2ec4c71 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:47.101292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5737c-b37b-4913-90f7-913ed37ed74e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:47.140530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11ba55-2821-42bc-93ef-fb830a7a3529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:47.141152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11ba55-2821-42bc-93ef-fb830a7a3529 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:47.141306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c75956-43aa-4795-aa64-22a315716f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:47.141435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11ba55-2821-42bc-93ef-fb830a7a3529 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:47.141713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11ba55-2821-42bc-93ef-fb830a7a3529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:47.141939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c75956-43aa-4795-aa64-22a315716f1e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:47.142224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c75956-43aa-4795-aa64-22a315716f1e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:47.142460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11ba55-2821-42bc-93ef-fb830a7a3529 None admin] RBAC: Adding query filter params (role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.domain.id=default, user.id=75ee7ae2d2194fbf9ae79851bb412a22) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:27:47.142553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c75956-43aa-4795-aa64-22a315716f1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:47.142774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11ba55-2821-42bc-93ef-fb830a7a3529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:47.143594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c75956-43aa-4795-aa64-22a315716f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:47.144240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11ba55-2821-42bc-93ef-fb830a7a3529 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:47.145126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c75956-43aa-4795-aa64-22a315716f1e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:47.146682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 23/44] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:27:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:47.151381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 22/45] 10.222.0.13 () {56 vars in 1322 bytes} [Thu Oct 20 14:27:46 2022] GET /identity/v3/role_assignments?user.id=75ee7ae2d2194fbf9ae79851bb412a22&scope.domain.id=default&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 530 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:47.152383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c75956-43aa-4795-aa64-22a315716f1e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:47.262658 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c75956-43aa-4795-aa64-22a315716f1e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:47.281000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:47.298634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a4a2f-d176-4ec9-ab60-fd1d98d307c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:47.299170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a4a2f-d176-4ec9-ab60-fd1d98d307c1 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:47.299455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a4a2f-d176-4ec9-ab60-fd1d98d307c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:47.299735 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a4a2f-d176-4ec9-ab60-fd1d98d307c1 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:47.300426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a4a2f-d176-4ec9-ab60-fd1d98d307c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:47.301889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a4a2f-d176-4ec9-ab60-fd1d98d307c1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:47.333021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 24/46] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:27:47 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:27:49.465378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfb511-581c-4c7a-9a83-b423bdac6987 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:49.466418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfb511-581c-4c7a-9a83-b423bdac6987 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:49.466418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfb511-581c-4c7a-9a83-b423bdac6987 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:49.467759 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 23/47] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:49.473026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee616f-3187-43a2-8667-b3a647a0104c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:49.473430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee616f-3187-43a2-8667-b3a647a0104c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:49.473587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee616f-3187-43a2-8667-b3a647a0104c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:49.491637 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-15ee616f-3187-43a2-8667-b3a647a0104c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:49.493104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-15ee616f-3187-43a2-8667-b3a647a0104c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:49.555856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 25/48] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:27:49.826388 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfb511-581c-4c7a-9a83-b423bdac6987 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:49.845039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:49.926274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfb511-581c-4c7a-9a83-b423bdac6987 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:49.961500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55774467-e7e0-420d-8d81-0e3387b7d516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:49.962124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774467-e7e0-420d-8d81-0e3387b7d516 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:49.962403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774467-e7e0-420d-8d81-0e3387b7d516 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:49.962678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774467-e7e0-420d-8d81-0e3387b7d516 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:49.963806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55774467-e7e0-420d-8d81-0e3387b7d516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:49.965279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55774467-e7e0-420d-8d81-0e3387b7d516 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:49.967438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 24/49] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:27:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:49.973120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee616f-3187-43a2-8667-b3a647a0104c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:50.066220 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:50.069030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee616f-3187-43a2-8667-b3a647a0104c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:50.103070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04682939-6a96-4830-bbce-f37845e055d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:50.103607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682939-6a96-4830-bbce-f37845e055d0 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:50.103885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682939-6a96-4830-bbce-f37845e055d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:50.104155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682939-6a96-4830-bbce-f37845e055d0 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:50.104829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04682939-6a96-4830-bbce-f37845e055d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:50.106860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04682939-6a96-4830-bbce-f37845e055d0 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:50.131039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 26/50] 10.222.0.13 () {60 vars in 1140 bytes} [Thu Oct 20 14:27:49 2022] POST /identity/v3/projects => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:27:52.267429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0291c-dae4-4157-bdfd-c2fe771f4c10 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:52.267943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0291c-dae4-4157-bdfd-c2fe771f4c10 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:52.268024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0291c-dae4-4157-bdfd-c2fe771f4c10 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:52.269357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 25/51] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:52.274628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9309c02-bc02-4368-b984-eda704e0d5bd None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:52.275018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9309c02-bc02-4368-b984-eda704e0d5bd None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:52.275190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9309c02-bc02-4368-b984-eda704e0d5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:52.293386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f9309c02-bc02-4368-b984-eda704e0d5bd None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:52.295253 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f9309c02-bc02-4368-b984-eda704e0d5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:52.358874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 27/52] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:27:52.631764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0291c-dae4-4157-bdfd-c2fe771f4c10 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:52.714781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:52.732327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0291c-dae4-4157-bdfd-c2fe771f4c10 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:52.765554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8711ff-e3d0-4eec-b0fe-dd1fb58ce5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:52.766177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8711ff-e3d0-4eec-b0fe-dd1fb58ce5d2 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:52.766463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8711ff-e3d0-4eec-b0fe-dd1fb58ce5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:52.766742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8711ff-e3d0-4eec-b0fe-dd1fb58ce5d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:52.767849 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8711ff-e3d0-4eec-b0fe-dd1fb58ce5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:52.769340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8711ff-e3d0-4eec-b0fe-dd1fb58ce5d2 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:52.770905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 26/53] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:27:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:52.776465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9309c02-bc02-4368-b984-eda704e0d5bd None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:52.853395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:52.871293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9309c02-bc02-4368-b984-eda704e0d5bd None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:52.904060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0698db0-b129-4cc2-a2aa-e46af9898e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:52.904621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0698db0-b129-4cc2-a2aa-e46af9898e49 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:52.904902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0698db0-b129-4cc2-a2aa-e46af9898e49 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:52.905177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0698db0-b129-4cc2-a2aa-e46af9898e49 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:52.905827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0698db0-b129-4cc2-a2aa-e46af9898e49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:52.907270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0698db0-b129-4cc2-a2aa-e46af9898e49 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:52.932227 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 28/54] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:27:52 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:27:56.312127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024364-a5cd-42d2-9d1b-440d8023ba77 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:56.313036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024364-a5cd-42d2-9d1b-440d8023ba77 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:56.314112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024364-a5cd-42d2-9d1b-440d8023ba77 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:56.315614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 27/55] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:56.321670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06e95-66df-4bcb-9a2a-580f4f5b74af None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:56.322269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06e95-66df-4bcb-9a2a-580f4f5b74af None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:56.322439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06e95-66df-4bcb-9a2a-580f4f5b74af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:56.349748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e4f06e95-66df-4bcb-9a2a-580f4f5b74af None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:56.351060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e4f06e95-66df-4bcb-9a2a-580f4f5b74af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:56.391353 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c7bca-8859-465b-860c-5d74c3a875f1 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:56.391973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c7bca-8859-465b-860c-5d74c3a875f1 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:56.392516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c7bca-8859-465b-860c-5d74c3a875f1 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:56.394547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 28/56] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:56.401200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb8c4-d7b1-42c4-ba17-e9dfc50e35c3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:56.401787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb8c4-d7b1-42c4-ba17-e9dfc50e35c3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:56.402277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb8c4-d7b1-42c4-ba17-e9dfc50e35c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:56.422986 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-433eb8c4-d7b1-42c4-ba17-e9dfc50e35c3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:56.424582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-433eb8c4-d7b1-42c4-ba17-e9dfc50e35c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:56.570533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 29/57] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:27:56.571618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 29/58] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:27:56.572240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37799a72-a87b-4202-9ae4-92e0d489165f None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:56.572585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37799a72-a87b-4202-9ae4-92e0d489165f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:56.572926 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37799a72-a87b-4202-9ae4-92e0d489165f None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:56.574233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 30/59] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:56.580015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9481216-7202-418f-90c9-118826e0e1c3 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:56.580541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9481216-7202-418f-90c9-118826e0e1c3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:56.580985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9481216-7202-418f-90c9-118826e0e1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:56.600877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c9481216-7202-418f-90c9-118826e0e1c3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:56.602331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c9481216-7202-418f-90c9-118826e0e1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:56.679501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:56.689126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 30/60] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:27:56.710765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfbce9-08bc-4d2d-9e81-5f93f356693a None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:56.711400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfbce9-08bc-4d2d-9e81-5f93f356693a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:56.711888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfbce9-08bc-4d2d-9e81-5f93f356693a None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:56.713615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 31/61] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:56.719845 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44003787-f014-4c8b-856d-21b6c78720d2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:56.720453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44003787-f014-4c8b-856d-21b6c78720d2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:56.720952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44003787-f014-4c8b-856d-21b6c78720d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:56.739435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-44003787-f014-4c8b-856d-21b6c78720d2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:56.740929 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-44003787-f014-4c8b-856d-21b6c78720d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:56.822939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 31/62] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:27:56.876641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfbce9-08bc-4d2d-9e81-5f93f356693a None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:56.900239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44003787-f014-4c8b-856d-21b6c78720d2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.037565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:57.050439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44003787-f014-4c8b-856d-21b6c78720d2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.082329 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:57.084173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfbce9-08bc-4d2d-9e81-5f93f356693a None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.093249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38a651-ebde-4195-87c3-ccf08963c0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.093995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38a651-ebde-4195-87c3-ccf08963c0d5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.094359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38a651-ebde-4195-87c3-ccf08963c0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.094707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38a651-ebde-4195-87c3-ccf08963c0d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.107341 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38a651-ebde-4195-87c3-ccf08963c0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-1d38a651-ebde-4195-87c3-ccf08963c0d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:27:57.113867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:27:57.114705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:57.115466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 32/63] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:27:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:27:57.116068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38a651-ebde-4195-87c3-ccf08963c0d5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.130770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa087fc-9e90-460e-b8d2-e71de2e55286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.132018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa087fc-9e90-460e-b8d2-e71de2e55286 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.132793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa087fc-9e90-460e-b8d2-e71de2e55286 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.133223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa087fc-9e90-460e-b8d2-e71de2e55286 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.147266 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa087fc-9e90-460e-b8d2-e71de2e55286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-2aa087fc-9e90-460e-b8d2-e71de2e55286 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:27:57.155352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:27:57.156251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:27:57.157110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:27:57.157667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 32/64] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:27:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:27:57.161060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa087fc-9e90-460e-b8d2-e71de2e55286 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.222512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38a651-ebde-4195-87c3-ccf08963c0d5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.255397 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdac49-2ec9-4b74-abd7-68eba131936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.256031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdac49-2ec9-4b74-abd7-68eba131936e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.256316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdac49-2ec9-4b74-abd7-68eba131936e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.256625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdac49-2ec9-4b74-abd7-68eba131936e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.260387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa087fc-9e90-460e-b8d2-e71de2e55286 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.266965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdac49-2ec9-4b74-abd7-68eba131936e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-eecdac49-2ec9-4b74-abd7-68eba131936e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:27:57.271850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:27:57.272695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:27:57.273410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 33/65] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:27:57.274695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdac49-2ec9-4b74-abd7-68eba131936e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.295994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcae6fe-3cbd-48d4-bdb2-a43d17b41822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.296658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcae6fe-3cbd-48d4-bdb2-a43d17b41822 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.296944 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcae6fe-3cbd-48d4-bdb2-a43d17b41822 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.297220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcae6fe-3cbd-48d4-bdb2-a43d17b41822 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.307384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcae6fe-3cbd-48d4-bdb2-a43d17b41822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-1fcae6fe-3cbd-48d4-bdb2-a43d17b41822 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:27:57.312555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:27:57.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 33/66] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:27:57.317412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcae6fe-3cbd-48d4-bdb2-a43d17b41822 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.318120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:57.329360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:57.380570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdac49-2ec9-4b74-abd7-68eba131936e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.414358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd180943-a009-41da-beab-d7e503e37065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.414969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180943-a009-41da-beab-d7e503e37065 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.415254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180943-a009-41da-beab-d7e503e37065 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.415539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180943-a009-41da-beab-d7e503e37065 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.416208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd180943-a009-41da-beab-d7e503e37065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:27:57.416547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd180943-a009-41da-beab-d7e503e37065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:57.417325 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcae6fe-3cbd-48d4-bdb2-a43d17b41822 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.417990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd180943-a009-41da-beab-d7e503e37065 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:57.426415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 34/67] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:57.427661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698e0ff-2002-4cb6-8647-4ec9ccb20f92 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.427957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698e0ff-2002-4cb6-8647-4ec9ccb20f92 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.428235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698e0ff-2002-4cb6-8647-4ec9ccb20f92 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.429461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 35/68] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:27:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:27:57.430169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698e0ff-2002-4cb6-8647-4ec9ccb20f92 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.457524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2bc1f-4451-432d-90ee-17f122d7e6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.458147 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2bc1f-4451-432d-90ee-17f122d7e6b3 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.458428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2bc1f-4451-432d-90ee-17f122d7e6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.458732 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2bc1f-4451-432d-90ee-17f122d7e6b3 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.459396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2bc1f-4451-432d-90ee-17f122d7e6b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:27:57.459704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2bc1f-4451-432d-90ee-17f122d7e6b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:57.461135 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2bc1f-4451-432d-90ee-17f122d7e6b3 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:57.469665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 34/69] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:57.470293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2bc1f-4451-432d-90ee-17f122d7e6b3 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.530950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698e0ff-2002-4cb6-8647-4ec9ccb20f92 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.534458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:57.541171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:57.564325 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011a938-7d96-467c-af4a-3078f991798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.564951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011a938-7d96-467c-af4a-3078f991798f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.565228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011a938-7d96-467c-af4a-3078f991798f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.565499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011a938-7d96-467c-af4a-3078f991798f None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.566179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011a938-7d96-467c-af4a-3078f991798f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:27:57.566478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011a938-7d96-467c-af4a-3078f991798f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:57.567844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011a938-7d96-467c-af4a-3078f991798f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:57.572285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2bc1f-4451-432d-90ee-17f122d7e6b3 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.576816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 36/70] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:57.581544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011a938-7d96-467c-af4a-3078f991798f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.606537 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1e7d0-9dec-4747-aa10-c5ae4283655b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.607083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1e7d0-9dec-4747-aa10-c5ae4283655b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.607367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1e7d0-9dec-4747-aa10-c5ae4283655b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.607653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1e7d0-9dec-4747-aa10-c5ae4283655b None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.608318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1e7d0-9dec-4747-aa10-c5ae4283655b None admin] RBAC: Adding query filter params (name=member) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:27:57.608731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1e7d0-9dec-4747-aa10-c5ae4283655b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:57.610113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1e7d0-9dec-4747-aa10-c5ae4283655b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:57.618166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 35/71] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:57.619414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39050c-35ce-41cc-a208-7ee96aa395f4 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.619705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39050c-35ce-41cc-a208-7ee96aa395f4 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.619987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39050c-35ce-41cc-a208-7ee96aa395f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.637769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-dd39050c-35ce-41cc-a208-7ee96aa395f4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:27:57.638989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-dd39050c-35ce-41cc-a208-7ee96aa395f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:27:57.679493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011a938-7d96-467c-af4a-3078f991798f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.704095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 36/72] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:27:57.704960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39050c-35ce-41cc-a208-7ee96aa395f4 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.722439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020855b-6019-4ac9-a558-11588b3dc071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.723075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020855b-6019-4ac9-a558-11588b3dc071 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.723362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020855b-6019-4ac9-a558-11588b3dc071 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.723640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020855b-6019-4ac9-a558-11588b3dc071 None admin] PATH_INFO: `/v3/users/75ee7ae2d2194fbf9ae79851bb412a22` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.724813 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020855b-6019-4ac9-a558-11588b3dc071 None admin] RBAC: Authorizing `identity:get_user(user_id=75ee7ae2d2194fbf9ae79851bb412a22)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:57.726452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020855b-6019-4ac9-a558-11588b3dc071 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:57.728604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 37/73] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/users/75ee7ae2d2194fbf9ae79851bb412a22 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:57.729202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020855b-6019-4ac9-a558-11588b3dc071 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.776005 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:57.778528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:57.820767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39050c-35ce-41cc-a208-7ee96aa395f4 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.840399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020855b-6019-4ac9-a558-11588b3dc071 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.856778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddac847-a624-4156-9c00-df3d9b686a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.857660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddac847-a624-4156-9c00-df3d9b686a12 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.858005 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddac847-a624-4156-9c00-df3d9b686a12 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.858258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddac847-a624-4156-9c00-df3d9b686a12 None admin] PATH_INFO: `/v3/users/29b4871936674640986d3796e5af8a0e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.873448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddac847-a624-4156-9c00-df3d9b686a12 None admin] RBAC: Authorizing `identity:get_user(user_id=29b4871936674640986d3796e5af8a0e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:57.874942 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddac847-a624-4156-9c00-df3d9b686a12 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:57.876607 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 37/74] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/users/29b4871936674640986d3796e5af8a0e => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:57.881280 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddac847-a624-4156-9c00-df3d9b686a12 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.882684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bc9dc-4392-47e8-bf15-9d4a15fb6462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:57.883601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bc9dc-4392-47e8-bf15-9d4a15fb6462 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:57.884765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bc9dc-4392-47e8-bf15-9d4a15fb6462 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:57.885052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bc9dc-4392-47e8-bf15-9d4a15fb6462 None admin] PATH_INFO: `/v3/users/29b4871936674640986d3796e5af8a0e` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:57.886199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bc9dc-4392-47e8-bf15-9d4a15fb6462 None admin] RBAC: Authorizing `identity:get_user(user_id=29b4871936674640986d3796e5af8a0e)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:57.887753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bc9dc-4392-47e8-bf15-9d4a15fb6462 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:57.889917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 38/75] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/users/29b4871936674640986d3796e5af8a0e => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:57.895009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bc9dc-4392-47e8-bf15-9d4a15fb6462 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:57.987749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddac847-a624-4156-9c00-df3d9b686a12 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:57.996234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bc9dc-4392-47e8-bf15-9d4a15fb6462 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.001582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:58.006032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:58.022512 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ff60b-6f43-4008-992c-23217ce5454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.023148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ff60b-6f43-4008-992c-23217ce5454f None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.023432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ff60b-6f43-4008-992c-23217ce5454f None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.023712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ff60b-6f43-4008-992c-23217ce5454f None admin] PATH_INFO: `/v3/users/29b4871936674640986d3796e5af8a0e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.024890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ff60b-6f43-4008-992c-23217ce5454f None admin] RBAC: Authorizing `identity:get_user(user_id=29b4871936674640986d3796e5af8a0e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.026344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ff60b-6f43-4008-992c-23217ce5454f None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.027948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 38/76] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/users/29b4871936674640986d3796e5af8a0e => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:58.029315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ff60b-6f43-4008-992c-23217ce5454f None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:58.035298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffb080-291c-48c4-826e-b5b247e59af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.035927 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffb080-291c-48c4-826e-b5b247e59af6 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.036208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffb080-291c-48c4-826e-b5b247e59af6 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.036515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffb080-291c-48c4-826e-b5b247e59af6 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.037639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffb080-291c-48c4-826e-b5b247e59af6 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.039089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffb080-291c-48c4-826e-b5b247e59af6 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.040776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 39/77] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:27:57 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:58.041523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffb080-291c-48c4-826e-b5b247e59af6 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:58.131665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ff60b-6f43-4008-992c-23217ce5454f None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.140752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffb080-291c-48c4-826e-b5b247e59af6 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.165159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da9b16-f19f-45d1-8e7f-8f06c029727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.165821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da9b16-f19f-45d1-8e7f-8f06c029727d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.166105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da9b16-f19f-45d1-8e7f-8f06c029727d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.166385 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da9b16-f19f-45d1-8e7f-8f06c029727d None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.167525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da9b16-f19f-45d1-8e7f-8f06c029727d None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.169004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da9b16-f19f-45d1-8e7f-8f06c029727d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.170636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 39/78] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:27:58 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:58.171399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da9b16-f19f-45d1-8e7f-8f06c029727d None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:58.176310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c71afe-0a3b-41cd-8f85-310db0ec409e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.176973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c71afe-0a3b-41cd-8f85-310db0ec409e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.177257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c71afe-0a3b-41cd-8f85-310db0ec409e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.177537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c71afe-0a3b-41cd-8f85-310db0ec409e None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.178659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c71afe-0a3b-41cd-8f85-310db0ec409e None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.180110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c71afe-0a3b-41cd-8f85-310db0ec409e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.181746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 40/79] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:27:58 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:58.182503 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c71afe-0a3b-41cd-8f85-310db0ec409e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:58.222277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:58.228460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:58.271318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da9b16-f19f-45d1-8e7f-8f06c029727d None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.283973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c71afe-0a3b-41cd-8f85-310db0ec409e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.303405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddcb19-4b50-4de8-953b-68a3b35ec095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.304024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddcb19-4b50-4de8-953b-68a3b35ec095 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.304305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddcb19-4b50-4de8-953b-68a3b35ec095 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.304605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddcb19-4b50-4de8-953b-68a3b35ec095 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.305715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddcb19-4b50-4de8-953b-68a3b35ec095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.307154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddcb19-4b50-4de8-953b-68a3b35ec095 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.308749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 40/80] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:27:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:58.310004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddcb19-4b50-4de8-953b-68a3b35ec095 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:58.319187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175616d-9722-41cd-8949-2526e632e8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.319817 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175616d-9722-41cd-8949-2526e632e8a4 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.320098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175616d-9722-41cd-8949-2526e632e8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.320411 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175616d-9722-41cd-8949-2526e632e8a4 None admin] PATH_INFO: `/v3/projects/d4571192766547cfb0e39a457b6deb42` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.321576 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175616d-9722-41cd-8949-2526e632e8a4 None admin] RBAC: Authorizing `identity:get_project(project_id=d4571192766547cfb0e39a457b6deb42)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.323053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175616d-9722-41cd-8949-2526e632e8a4 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.324713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 41/81] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:27:58 2022] GET /identity/v3/projects/d4571192766547cfb0e39a457b6deb42 => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:27:58.330071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175616d-9722-41cd-8949-2526e632e8a4 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:58.410235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddcb19-4b50-4de8-953b-68a3b35ec095 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.425995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175616d-9722-41cd-8949-2526e632e8a4 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.427577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:58.439329 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:58.444062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7231c-d914-4fb1-b616-0585e7f95fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.444746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7231c-d914-4fb1-b616-0585e7f95fb0 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.445027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7231c-d914-4fb1-b616-0585e7f95fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.445303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7231c-d914-4fb1-b616-0585e7f95fb0 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/users/75ee7ae2d2194fbf9ae79851bb412a22/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.447206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7231c-d914-4fb1-b616-0585e7f95fb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dde0c3629144eabb2ec9e6e6c16a81, user_id=75ee7ae2d2194fbf9ae79851bb412a22, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.448716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7231c-d914-4fb1-b616-0585e7f95fb0 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.460039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243736d-8e42-4394-858c-3d56baf4f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.460716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243736d-8e42-4394-858c-3d56baf4f733 None admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.460998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243736d-8e42-4394-858c-3d56baf4f733 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.461102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 41/82] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:27:58 2022] PUT /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/users/75ee7ae2d2194fbf9ae79851bb412a22/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:27:58.461275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243736d-8e42-4394-858c-3d56baf4f733 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/users/29b4871936674640986d3796e5af8a0e/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.461835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7231c-d914-4fb1-b616-0585e7f95fb0 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:58.467467 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243736d-8e42-4394-858c-3d56baf4f733 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dde0c3629144eabb2ec9e6e6c16a81, user_id=29b4871936674640986d3796e5af8a0e, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.469032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243736d-8e42-4394-858c-3d56baf4f733 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.489089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 42/83] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:27:58 2022] PUT /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/users/29b4871936674640986d3796e5af8a0e/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:27:58.489754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243736d-8e42-4394-858c-3d56baf4f733 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:58.601052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7231c-d914-4fb1-b616-0585e7f95fb0 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.626816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243736d-8e42-4394-858c-3d56baf4f733 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.654874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321ce06-e7eb-4476-9d47-cb4a3316b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.656794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321ce06-e7eb-4476-9d47-cb4a3316b297 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.657608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321ce06-e7eb-4476-9d47-cb4a3316b297 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.658102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321ce06-e7eb-4476-9d47-cb4a3316b297 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/users/29b4871936674640986d3796e5af8a0e/roles/dff349185bc94cb4a0e0f0c40425862f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.662157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321ce06-e7eb-4476-9d47-cb4a3316b297 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dde0c3629144eabb2ec9e6e6c16a81, user_id=29b4871936674640986d3796e5af8a0e, role_id=dff349185bc94cb4a0e0f0c40425862f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.665392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321ce06-e7eb-4476-9d47-cb4a3316b297 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.674682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058d910-5245-4b32-8f66-506a5af87f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.675434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058d910-5245-4b32-8f66-506a5af87f9c None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.675789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058d910-5245-4b32-8f66-506a5af87f9c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.676319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058d910-5245-4b32-8f66-506a5af87f9c None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.677268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058d910-5245-4b32-8f66-506a5af87f9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.678872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058d910-5245-4b32-8f66-506a5af87f9c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.687228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 42/84] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:27:58 2022] PUT /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/users/29b4871936674640986d3796e5af8a0e/roles/dff349185bc94cb4a0e0f0c40425862f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:27:58.688584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321ce06-e7eb-4476-9d47-cb4a3316b297 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:27:58.709051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 43/85] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:27:58 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:27:58.744887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:27:58.853175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321ce06-e7eb-4476-9d47-cb4a3316b297 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:27:58.910952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ead35-3cfb-420d-a40b-65dce82a7145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:27:58.912018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ead35-3cfb-420d-a40b-65dce82a7145 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:27:58.912678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ead35-3cfb-420d-a40b-65dce82a7145 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:27:58.913179 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ead35-3cfb-420d-a40b-65dce82a7145 None admin] PATH_INFO: `/v3/projects/d4571192766547cfb0e39a457b6deb42/users/29b4871936674640986d3796e5af8a0e/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:27:58.915614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ead35-3cfb-420d-a40b-65dce82a7145 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4571192766547cfb0e39a457b6deb42, user_id=29b4871936674640986d3796e5af8a0e, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:27:58.918548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ead35-3cfb-420d-a40b-65dce82a7145 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:27:58.940454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 43/86] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:27:58 2022] PUT /identity/v3/projects/d4571192766547cfb0e39a457b6deb42/users/29b4871936674640986d3796e5af8a0e/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:01.298982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a2da4-dcb2-4fee-9ff7-2d3cbfa69a6f None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:01.299499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a2da4-dcb2-4fee-9ff7-2d3cbfa69a6f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:01.299571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a2da4-dcb2-4fee-9ff7-2d3cbfa69a6f None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:01.300943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 44/87] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:01.306886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77825692-81f2-432e-a414-1bbd40a83887 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:01.307375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77825692-81f2-432e-a414-1bbd40a83887 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:01.307445 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77825692-81f2-432e-a414-1bbd40a83887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:01.327965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-77825692-81f2-432e-a414-1bbd40a83887 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:01.329221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-77825692-81f2-432e-a414-1bbd40a83887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:01.391271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce757ec4-aa30-48f9-9843-95b29188dfd0 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:01.391561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce757ec4-aa30-48f9-9843-95b29188dfd0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:01.392080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce757ec4-aa30-48f9-9843-95b29188dfd0 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:01.393933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 45/88] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:01.398655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ff0d8-da6f-40c8-ba0f-300f3e1262e4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:01.399194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ff0d8-da6f-40c8-ba0f-300f3e1262e4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:01.404569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ff0d8-da6f-40c8-ba0f-300f3e1262e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:01.431166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 44/89] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:01.437804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-211ff0d8-da6f-40c8-ba0f-300f3e1262e4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:01.439158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-211ff0d8-da6f-40c8-ba0f-300f3e1262e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:01.468772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:01.541684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 46/90] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:01.820614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22547880-86b6-4abd-886a-7be78a294461 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:01.821309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22547880-86b6-4abd-886a-7be78a294461 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:01.821799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22547880-86b6-4abd-886a-7be78a294461 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:01.823651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 45/91] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:01.829431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1567de9-b670-4782-90cf-84f8f9f8a919 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:01.829955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1567de9-b670-4782-90cf-84f8f9f8a919 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:01.830452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1567de9-b670-4782-90cf-84f8f9f8a919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:01.850312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-b1567de9-b670-4782-90cf-84f8f9f8a919 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:01.851840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b1567de9-b670-4782-90cf-84f8f9f8a919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:01.946496 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 47/92] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:01.968257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22547880-86b6-4abd-886a-7be78a294461 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:02.006960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:02.017532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1567de9-b670-4782-90cf-84f8f9f8a919 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:02.121054 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:02.129941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22547880-86b6-4abd-886a-7be78a294461 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:02.147035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1567de9-b670-4782-90cf-84f8f9f8a919 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:02.194213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeafac4-176d-43ff-b2be-dde1bdd438f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:02.194858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeafac4-176d-43ff-b2be-dde1bdd438f6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.195146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeafac4-176d-43ff-b2be-dde1bdd438f6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.195428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeafac4-176d-43ff-b2be-dde1bdd438f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.210490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59d34b-62a6-4b45-8809-e28ea0f082bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:02.211343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59d34b-62a6-4b45-8809-e28ea0f082bd None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.211770 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59d34b-62a6-4b45-8809-e28ea0f082bd None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.212186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59d34b-62a6-4b45-8809-e28ea0f082bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.216413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeafac4-176d-43ff-b2be-dde1bdd438f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-adeafac4-176d-43ff-b2be-dde1bdd438f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:02.222449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:02.223489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:02.224464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:02.228499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 46/93] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:02.229964 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723de9cf-ea06-4e70-ad2c-3080cdd40ab5 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.231778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723de9cf-ea06-4e70-ad2c-3080cdd40ab5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.232447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59d34b-62a6-4b45-8809-e28ea0f082bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:02.236396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723de9cf-ea06-4e70-ad2c-3080cdd40ab5 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.237854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 47/94] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-2d59d34b-62a6-4b45-8809-e28ea0f082bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:02.239452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:02.240488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:02.241369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:02.242942 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723de9cf-ea06-4e70-ad2c-3080cdd40ab5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:02.244634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 48/95] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:28:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:02.247969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f05b5-f1c1-422a-b653-af23f44ce952 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.249358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f05b5-f1c1-422a-b653-af23f44ce952 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.250595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f05b5-f1c1-422a-b653-af23f44ce952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.278869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-b68f05b5-f1c1-422a-b653-af23f44ce952 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:02.280532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b68f05b5-f1c1-422a-b653-af23f44ce952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:02.358036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 49/96] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:02.359071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f05b5-f1c1-422a-b653-af23f44ce952 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:02.395931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:02.419364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723de9cf-ea06-4e70-ad2c-3080cdd40ab5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:02.465487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4e97c-029d-4c4b-b43e-cf77821e1b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:02.467328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4e97c-029d-4c4b-b43e-cf77821e1b98 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.467835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4e97c-029d-4c4b-b43e-cf77821e1b98 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.468318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4e97c-029d-4c4b-b43e-cf77821e1b98 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.469595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4e97c-029d-4c4b-b43e-cf77821e1b98 None admin] RBAC: Adding query filter params (name=member) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:02.470121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4e97c-029d-4c4b-b43e-cf77821e1b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:02.471946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4e97c-029d-4c4b-b43e-cf77821e1b98 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:02.484366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 48/97] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:02.485556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4e97c-029d-4c4b-b43e-cf77821e1b98 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:02.490539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:02.536613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f05b5-f1c1-422a-b653-af23f44ce952 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:02.572288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfc170-371a-473c-978b-cf3fea5decb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:02.573422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfc170-371a-473c-978b-cf3fea5decb6 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.573422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfc170-371a-473c-978b-cf3fea5decb6 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.573585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfc170-371a-473c-978b-cf3fea5decb6 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.574142 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfc170-371a-473c-978b-cf3fea5decb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:02.574455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfc170-371a-473c-978b-cf3fea5decb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:02.576499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfc170-371a-473c-978b-cf3fea5decb6 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:02.584739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 50/98] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:02.585541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfc170-371a-473c-978b-cf3fea5decb6 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:02.609256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4e97c-029d-4c4b-b43e-cf77821e1b98 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:02.647568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb21e9-fbbe-470c-96cd-d00fe8f9fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:02.648134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb21e9-fbbe-470c-96cd-d00fe8f9fe55 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.648458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb21e9-fbbe-470c-96cd-d00fe8f9fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.648740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb21e9-fbbe-470c-96cd-d00fe8f9fe55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.660335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb21e9-fbbe-470c-96cd-d00fe8f9fe55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-95eb21e9-fbbe-470c-96cd-d00fe8f9fe55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:02.665904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:02.666801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:02.667543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 49/99] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:28:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:02.668330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e38aa-d7e4-45b1-8b90-613cc3abe00a None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.669697 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e38aa-d7e4-45b1-8b90-613cc3abe00a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.669988 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e38aa-d7e4-45b1-8b90-613cc3abe00a None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.671494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 50/100] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:02.672629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a04702-7ef6-4a6e-b4b2-e1aacbcd7cc7 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.672925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a04702-7ef6-4a6e-b4b2-e1aacbcd7cc7 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.673203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a04702-7ef6-4a6e-b4b2-e1aacbcd7cc7 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.674376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 51/101] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:02.675069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a04702-7ef6-4a6e-b4b2-e1aacbcd7cc7 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:02.692508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:02.701604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfc170-371a-473c-978b-cf3fea5decb6 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:02.735731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492f30f-58e8-4ca0-b6a7-a173e44cf42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:02.736384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f30f-58e8-4ca0-b6a7-a173e44cf42c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.736673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f30f-58e8-4ca0-b6a7-a173e44cf42c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.736952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f30f-58e8-4ca0-b6a7-a173e44cf42c None admin] PATH_INFO: `/v3/users/29b4871936674640986d3796e5af8a0e` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.738087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492f30f-58e8-4ca0-b6a7-a173e44cf42c None admin] RBAC: Authorizing `identity:get_user(user_id=29b4871936674640986d3796e5af8a0e)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:02.739531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492f30f-58e8-4ca0-b6a7-a173e44cf42c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:02.741144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 51/102] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:02 2022] GET /identity/v3/users/29b4871936674640986d3796e5af8a0e => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:02.741144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492f30f-58e8-4ca0-b6a7-a173e44cf42c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:02.757484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:02.785371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a04702-7ef6-4a6e-b4b2-e1aacbcd7cc7 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:02.821530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2509c46-3c88-473d-aa67-abf0c231fd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:02.822183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2509c46-3c88-473d-aa67-abf0c231fd7c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.823140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2509c46-3c88-473d-aa67-abf0c231fd7c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.823140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2509c46-3c88-473d-aa67-abf0c231fd7c None admin] PATH_INFO: `/v3/users/29b4871936674640986d3796e5af8a0e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.824711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2509c46-3c88-473d-aa67-abf0c231fd7c None admin] RBAC: Authorizing `identity:get_user(user_id=29b4871936674640986d3796e5af8a0e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:02.827151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2509c46-3c88-473d-aa67-abf0c231fd7c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:02.828787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 52/103] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:02 2022] GET /identity/v3/users/29b4871936674640986d3796e5af8a0e => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:02.829963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f610e7-0a85-4214-a11a-3c9deea19192 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.830252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f610e7-0a85-4214-a11a-3c9deea19192 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.831220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f610e7-0a85-4214-a11a-3c9deea19192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.848892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-a0f610e7-0a85-4214-a11a-3c9deea19192 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:02.850075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a0f610e7-0a85-4214-a11a-3c9deea19192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:02.853173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492f30f-58e8-4ca0-b6a7-a173e44cf42c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:02.886443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec940b7-ce3f-42d7-9814-f72a5e628a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:02.887128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec940b7-ce3f-42d7-9814-f72a5e628a7b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.888074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec940b7-ce3f-42d7-9814-f72a5e628a7b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.888459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec940b7-ce3f-42d7-9814-f72a5e628a7b None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.889216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec940b7-ce3f-42d7-9814-f72a5e628a7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:02.889597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec940b7-ce3f-42d7-9814-f72a5e628a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:02.891042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec940b7-ce3f-42d7-9814-f72a5e628a7b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:02.899113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 52/104] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:02.900517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e3269-7cc7-490d-afb9-f0664db963a6 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:02.900805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e3269-7cc7-490d-afb9-f0664db963a6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:02.901081 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e3269-7cc7-490d-afb9-f0664db963a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:02.916207 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 53/105] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:02.916939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f610e7-0a85-4214-a11a-3c9deea19192 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:02.923198 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-978e3269-7cc7-490d-afb9-f0664db963a6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:02.924412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-978e3269-7cc7-490d-afb9-f0664db963a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:02.927989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:02.991397 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:02.997270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 53/106] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:02.998371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e3269-7cc7-490d-afb9-f0664db963a6 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.038260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f610e7-0a85-4214-a11a-3c9deea19192 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.089961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f8d62-4ec5-4f92-b5e9-6ac74823f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:03.096375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f8d62-4ec5-4f92-b5e9-6ac74823f83a None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:03.096375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f8d62-4ec5-4f92-b5e9-6ac74823f83a None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:03.096375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f8d62-4ec5-4f92-b5e9-6ac74823f83a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:03.102865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f8d62-4ec5-4f92-b5e9-6ac74823f83a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-5b6f8d62-4ec5-4f92-b5e9-6ac74823f83a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:03.109411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:03.110231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:03.110978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 54/107] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:03.111585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f8d62-4ec5-4f92-b5e9-6ac74823f83a None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.175105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e3269-7cc7-490d-afb9-f0664db963a6 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.193397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:03.232376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddb44d-d702-4677-b98a-296eb07e290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:03.232376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddb44d-d702-4677-b98a-296eb07e290b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:03.232925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddb44d-d702-4677-b98a-296eb07e290b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:03.233415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddb44d-d702-4677-b98a-296eb07e290b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:03.233922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f8d62-4ec5-4f92-b5e9-6ac74823f83a None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.273631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969037ca-6249-4a31-8be3-b3c23d58235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:03.274349 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969037ca-6249-4a31-8be3-b3c23d58235b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:03.274634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969037ca-6249-4a31-8be3-b3c23d58235b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:03.274910 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969037ca-6249-4a31-8be3-b3c23d58235b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:03.313833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:03.316493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddb44d-d702-4677-b98a-296eb07e290b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:03.318329 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddb44d-d702-4677-b98a-296eb07e290b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:03.347309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969037ca-6249-4a31-8be3-b3c23d58235b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:03.348862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969037ca-6249-4a31-8be3-b3c23d58235b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:03.416065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:03.435780 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 54/108] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:03.436578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddb44d-d702-4677-b98a-296eb07e290b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.455819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 55/109] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:03.455819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969037ca-6249-4a31-8be3-b3c23d58235b None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.539690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:03.542475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddb44d-d702-4677-b98a-296eb07e290b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.555026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969037ca-6249-4a31-8be3-b3c23d58235b None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.577019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfde71-fc4e-4eb4-910f-b94f94d2970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:03.577643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfde71-fc4e-4eb4-910f-b94f94d2970d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:03.577925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfde71-fc4e-4eb4-910f-b94f94d2970d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:03.578203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfde71-fc4e-4eb4-910f-b94f94d2970d None admin] PATH_INFO: `/v3/users/75ee7ae2d2194fbf9ae79851bb412a22` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:03.579312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfde71-fc4e-4eb4-910f-b94f94d2970d None admin] RBAC: Authorizing `identity:get_user(user_id=75ee7ae2d2194fbf9ae79851bb412a22)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:03.580782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfde71-fc4e-4eb4-910f-b94f94d2970d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:03.582537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 55/110] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:03 2022] GET /identity/v3/users/75ee7ae2d2194fbf9ae79851bb412a22 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:03.583174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfde71-fc4e-4eb4-910f-b94f94d2970d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.593163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627edb54-8e1a-4836-9bc5-eac06dd08ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:03.593775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627edb54-8e1a-4836-9bc5-eac06dd08ab6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:03.594055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627edb54-8e1a-4836-9bc5-eac06dd08ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:03.594329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627edb54-8e1a-4836-9bc5-eac06dd08ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:03.595002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627edb54-8e1a-4836-9bc5-eac06dd08ab6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:03.595312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627edb54-8e1a-4836-9bc5-eac06dd08ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:03.596721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627edb54-8e1a-4836-9bc5-eac06dd08ab6 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:03.604961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 56/111] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:03.605741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627edb54-8e1a-4836-9bc5-eac06dd08ab6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.635548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:03.679457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfde71-fc4e-4eb4-910f-b94f94d2970d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.704590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627edb54-8e1a-4836-9bc5-eac06dd08ab6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.712479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fbcd2-dd00-4505-8cd1-27cd2e7bc72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:03.713107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fbcd2-dd00-4505-8cd1-27cd2e7bc72a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:03.713388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fbcd2-dd00-4505-8cd1-27cd2e7bc72a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:03.713666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fbcd2-dd00-4505-8cd1-27cd2e7bc72a None admin] PATH_INFO: `/v3/roles/member` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:03.724322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fbcd2-dd00-4505-8cd1-27cd2e7bc72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-ef9fbcd2-dd00-4505-8cd1-27cd2e7bc72a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:03.729219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:03.730182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:03.731173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 56/112] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:03.731779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fbcd2-dd00-4505-8cd1-27cd2e7bc72a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.743644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846630c3-8bf4-4cda-81a3-adf804498c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:03.744276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846630c3-8bf4-4cda-81a3-adf804498c0e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:03.744584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846630c3-8bf4-4cda-81a3-adf804498c0e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:03.744860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846630c3-8bf4-4cda-81a3-adf804498c0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:03.745952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846630c3-8bf4-4cda-81a3-adf804498c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:03.748180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846630c3-8bf4-4cda-81a3-adf804498c0e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:03.749902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 57/113] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:03.750592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846630c3-8bf4-4cda-81a3-adf804498c0e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.767743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:03.840535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fbcd2-dd00-4505-8cd1-27cd2e7bc72a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.846420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:03.850257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846630c3-8bf4-4cda-81a3-adf804498c0e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.874459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21315fc1-8544-410e-8910-576fdaddc16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:03.875086 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21315fc1-8544-410e-8910-576fdaddc16b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:03.875367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21315fc1-8544-410e-8910-576fdaddc16b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:03.875641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21315fc1-8544-410e-8910-576fdaddc16b None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:03.876774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21315fc1-8544-410e-8910-576fdaddc16b None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:03.878223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21315fc1-8544-410e-8910-576fdaddc16b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:03.879828 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 57/114] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:03 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:03.880599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21315fc1-8544-410e-8910-576fdaddc16b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.885337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3151f-2df8-48b4-ad75-7815e87a46a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:03.885980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3151f-2df8-48b4-ad75-7815e87a46a7 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:03.886267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3151f-2df8-48b4-ad75-7815e87a46a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:03.886550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3151f-2df8-48b4-ad75-7815e87a46a7 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:03.887874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3151f-2df8-48b4-ad75-7815e87a46a7 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:03.889360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3151f-2df8-48b4-ad75-7815e87a46a7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:03.891052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 58/115] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:03 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:03.891846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3151f-2df8-48b4-ad75-7815e87a46a7 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:03.976531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:03.983195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21315fc1-8544-410e-8910-576fdaddc16b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:03.988865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3151f-2df8-48b4-ad75-7815e87a46a7 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.019252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e953f-e77d-4eeb-add1-609bfa3b9146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.019849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e953f-e77d-4eeb-add1-609bfa3b9146 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.020127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e953f-e77d-4eeb-add1-609bfa3b9146 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.020432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e953f-e77d-4eeb-add1-609bfa3b9146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.022047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659d676-028f-45cb-a1b7-f118c7ff06ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.022683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659d676-028f-45cb-a1b7-f118c7ff06ab None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.022970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659d676-028f-45cb-a1b7-f118c7ff06ab None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.023252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659d676-028f-45cb-a1b7-f118c7ff06ab None admin] PATH_INFO: `/v3/users/8f9d2f52dec345548775b353df4faccd` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.036326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659d676-028f-45cb-a1b7-f118c7ff06ab None admin] RBAC: Authorizing `identity:get_user(user_id=8f9d2f52dec345548775b353df4faccd)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.037843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659d676-028f-45cb-a1b7-f118c7ff06ab None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.039463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 59/116] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:03 2022] GET /identity/v3/users/8f9d2f52dec345548775b353df4faccd => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:04.040249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659d676-028f-45cb-a1b7-f118c7ff06ab None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.065281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:04.088926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e953f-e77d-4eeb-add1-609bfa3b9146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.090545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e953f-e77d-4eeb-add1-609bfa3b9146 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.138870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659d676-028f-45cb-a1b7-f118c7ff06ab None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.173046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d6654-6dce-4f2d-915a-92f089271b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.173651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d6654-6dce-4f2d-915a-92f089271b54 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.173935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d6654-6dce-4f2d-915a-92f089271b54 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.174213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d6654-6dce-4f2d-915a-92f089271b54 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.174877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d6654-6dce-4f2d-915a-92f089271b54 None admin] RBAC: Adding query filter params (name=member) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:04.175187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d6654-6dce-4f2d-915a-92f089271b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.176614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d6654-6dce-4f2d-915a-92f089271b54 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.182169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:04.185014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 60/117] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:04.189200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d6654-6dce-4f2d-915a-92f089271b54 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.193956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 58/118] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:04.193956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e953f-e77d-4eeb-add1-609bfa3b9146 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.273899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:04.288736 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d6654-6dce-4f2d-915a-92f089271b54 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.289105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e953f-e77d-4eeb-add1-609bfa3b9146 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.321627 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf385854-2fd4-4afd-ac56-893ebcaacd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.321810 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee88e1-f8b7-4ea5-a6ff-723ae14d4fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.322250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf385854-2fd4-4afd-ac56-893ebcaacd71 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.322375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee88e1-f8b7-4ea5-a6ff-723ae14d4fda None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.322529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf385854-2fd4-4afd-ac56-893ebcaacd71 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.322661 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee88e1-f8b7-4ea5-a6ff-723ae14d4fda None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.322801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf385854-2fd4-4afd-ac56-893ebcaacd71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.322939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee88e1-f8b7-4ea5-a6ff-723ae14d4fda None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.323519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf385854-2fd4-4afd-ac56-893ebcaacd71 None admin] RBAC: Adding query filter params (role.id=a73215dcd05645c89a2f742200bbb0c9, scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81, user.id=29b4871936674640986d3796e5af8a0e) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:04.323603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee88e1-f8b7-4ea5-a6ff-723ae14d4fda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.323828 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf385854-2fd4-4afd-ac56-893ebcaacd71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.325736 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee88e1-f8b7-4ea5-a6ff-723ae14d4fda None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.326039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf385854-2fd4-4afd-ac56-893ebcaacd71 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.334099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 59/119] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:04 2022] GET /identity/v3/role_assignments?user.id=29b4871936674640986d3796e5af8a0e&scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81&role.id=a73215dcd05645c89a2f742200bbb0c9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:04.334903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf385854-2fd4-4afd-ac56-893ebcaacd71 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.358835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 61/120] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:28:04 2022] POST /identity/v3/projects => generated 302 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:04.360076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee88e1-f8b7-4ea5-a6ff-723ae14d4fda None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.424483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:04.481972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf385854-2fd4-4afd-ac56-893ebcaacd71 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.493114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee88e1-f8b7-4ea5-a6ff-723ae14d4fda None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.536199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73d756-4486-43e9-9d80-4a6245b8f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.537162 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73d756-4486-43e9-9d80-4a6245b8f94f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.537528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73d756-4486-43e9-9d80-4a6245b8f94f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.537811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73d756-4486-43e9-9d80-4a6245b8f94f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.538594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73d756-4486-43e9-9d80-4a6245b8f94f None admin] RBAC: Adding query filter params (role.id=dff349185bc94cb4a0e0f0c40425862f, scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81, user.id=29b4871936674640986d3796e5af8a0e) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:04.538912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73d756-4486-43e9-9d80-4a6245b8f94f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.540439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73d756-4486-43e9-9d80-4a6245b8f94f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.544847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaaacfb-54f7-42ab-bf15-36d6b3aded17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.545502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaacfb-54f7-42ab-bf15-36d6b3aded17 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.545792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaacfb-54f7-42ab-bf15-36d6b3aded17 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.546077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaacfb-54f7-42ab-bf15-36d6b3aded17 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.547314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaaacfb-54f7-42ab-bf15-36d6b3aded17 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.548808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaaacfb-54f7-42ab-bf15-36d6b3aded17 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.549003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 60/121] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:04 2022] GET /identity/v3/role_assignments?user.id=29b4871936674640986d3796e5af8a0e&scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81&role.id=dff349185bc94cb4a0e0f0c40425862f => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:04.549843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73d756-4486-43e9-9d80-4a6245b8f94f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.551830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 62/122] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:04 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:04.552584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaaacfb-54f7-42ab-bf15-36d6b3aded17 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.563330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:04.693745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaaacfb-54f7-42ab-bf15-36d6b3aded17 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.694217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73d756-4486-43e9-9d80-4a6245b8f94f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.721336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:04.733879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe92e6-d71a-444c-b99f-5de7117bb35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.734685 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06b328-ebec-4191-9816-b10454c161f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.734806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe92e6-d71a-444c-b99f-5de7117bb35c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.735152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe92e6-d71a-444c-b99f-5de7117bb35c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.735336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06b328-ebec-4191-9816-b10454c161f6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.735613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe92e6-d71a-444c-b99f-5de7117bb35c None admin] PATH_INFO: `/v3/users/29b4871936674640986d3796e5af8a0e` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.735715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06b328-ebec-4191-9816-b10454c161f6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.735893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06b328-ebec-4191-9816-b10454c161f6 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.737065 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe92e6-d71a-444c-b99f-5de7117bb35c None admin] RBAC: Authorizing `identity:get_user(user_id=29b4871936674640986d3796e5af8a0e)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.737177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06b328-ebec-4191-9816-b10454c161f6 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.738557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06b328-ebec-4191-9816-b10454c161f6 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.738805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe92e6-d71a-444c-b99f-5de7117bb35c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.740593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 61/124] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:04 2022] GET /identity/v3/users/29b4871936674640986d3796e5af8a0e => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:04.741493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe92e6-d71a-444c-b99f-5de7117bb35c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.745677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 63/124] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:04 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:04.747731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06b328-ebec-4191-9816-b10454c161f6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.855571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:04.866763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe92e6-d71a-444c-b99f-5de7117bb35c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.888952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06b328-ebec-4191-9816-b10454c161f6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:04.907153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526408ae-e0a9-4031-8e4f-eadffa20c5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.907909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526408ae-e0a9-4031-8e4f-eadffa20c5df None admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.908264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526408ae-e0a9-4031-8e4f-eadffa20c5df None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.908652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526408ae-e0a9-4031-8e4f-eadffa20c5df None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/users/8f9d2f52dec345548775b353df4faccd/roles/c8d552ca72b140a8a815448527770138` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.910684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526408ae-e0a9-4031-8e4f-eadffa20c5df None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dde0c3629144eabb2ec9e6e6c16a81, user_id=8f9d2f52dec345548775b353df4faccd, role_id=c8d552ca72b140a8a815448527770138)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:04.912232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526408ae-e0a9-4031-8e4f-eadffa20c5df None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:04.925520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd43ba-56ee-4232-840d-d39a9ff60f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:04.925711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 62/125] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:04 2022] PUT /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/users/8f9d2f52dec345548775b353df4faccd/roles/c8d552ca72b140a8a815448527770138 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:04.929989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd43ba-56ee-4232-840d-d39a9ff60f64 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:04.929989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd43ba-56ee-4232-840d-d39a9ff60f64 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:04.930202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526408ae-e0a9-4031-8e4f-eadffa20c5df None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:04.930295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd43ba-56ee-4232-840d-d39a9ff60f64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:04.990001 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:05.034255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd43ba-56ee-4232-840d-d39a9ff60f64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:05.038850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd43ba-56ee-4232-840d-d39a9ff60f64 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:05.052844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526408ae-e0a9-4031-8e4f-eadffa20c5df None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:05.094776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f07f5-ee40-427d-bcb0-871fdf9250f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:05.095702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f07f5-ee40-427d-bcb0-871fdf9250f9 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:05.096213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f07f5-ee40-427d-bcb0-871fdf9250f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:05.096785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f07f5-ee40-427d-bcb0-871fdf9250f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:05.097766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f07f5-ee40-427d-bcb0-871fdf9250f9 None admin] RBAC: Adding query filter params (role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81, user.id=75ee7ae2d2194fbf9ae79851bb412a22) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:05.098431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f07f5-ee40-427d-bcb0-871fdf9250f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:05.100234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f07f5-ee40-427d-bcb0-871fdf9250f9 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:05.108194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 63/126] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:04 2022] GET /identity/v3/role_assignments?user.id=75ee7ae2d2194fbf9ae79851bb412a22&scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:05.140000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:05.170720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 64/127] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 20 14:28:05.176856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f07f5-ee40-427d-bcb0-871fdf9250f9 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:05.291273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f07f5-ee40-427d-bcb0-871fdf9250f9 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:05.308755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:05.329124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608247fb-de62-4608-b88f-9900fb8aff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:05.330054 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608247fb-de62-4608-b88f-9900fb8aff30 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:05.330515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608247fb-de62-4608-b88f-9900fb8aff30 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:05.330997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608247fb-de62-4608-b88f-9900fb8aff30 None admin] PATH_INFO: `/v3/projects/d4571192766547cfb0e39a457b6deb42` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:05.332405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608247fb-de62-4608-b88f-9900fb8aff30 None admin] RBAC: Authorizing `identity:get_project(project_id=d4571192766547cfb0e39a457b6deb42)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:05.334105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608247fb-de62-4608-b88f-9900fb8aff30 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:05.336228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 64/128] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:05 2022] GET /identity/v3/projects/d4571192766547cfb0e39a457b6deb42 => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:05.347248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd43ba-56ee-4232-840d-d39a9ff60f64 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:05.461574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd43ba-56ee-4232-840d-d39a9ff60f64 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:05.496148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eeb0f7-e4f9-47a3-b7fc-80ee471eb613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:05.497801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eeb0f7-e4f9-47a3-b7fc-80ee471eb613 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:05.498101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eeb0f7-e4f9-47a3-b7fc-80ee471eb613 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:05.498368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eeb0f7-e4f9-47a3-b7fc-80ee471eb613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:05.499127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eeb0f7-e4f9-47a3-b7fc-80ee471eb613 None admin] RBAC: Adding query filter params (role.id=a73215dcd05645c89a2f742200bbb0c9, scope.project.id=d4571192766547cfb0e39a457b6deb42, user.id=29b4871936674640986d3796e5af8a0e) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:05.499440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eeb0f7-e4f9-47a3-b7fc-80ee471eb613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:05.500931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eeb0f7-e4f9-47a3-b7fc-80ee471eb613 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:05.506645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 65/129] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:05 2022] GET /identity/v3/role_assignments?user.id=29b4871936674640986d3796e5af8a0e&scope.project.id=d4571192766547cfb0e39a457b6deb42&role.id=a73215dcd05645c89a2f742200bbb0c9 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:06.657876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41a027-f951-4b25-bc66-c83df48263da None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:06.658359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41a027-f951-4b25-bc66-c83df48263da None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:06.658359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41a027-f951-4b25-bc66-c83df48263da None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:06.659695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 65/130] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:06.665012 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97802ea0-ac2d-4741-b380-52267b86f5b6 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:06.665422 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97802ea0-ac2d-4741-b380-52267b86f5b6 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:06.665563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97802ea0-ac2d-4741-b380-52267b86f5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:06.683642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-97802ea0-ac2d-4741-b380-52267b86f5b6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:06.684925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-97802ea0-ac2d-4741-b380-52267b86f5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:06.712717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:06.751516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 66/131] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:07.027924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41a027-f951-4b25-bc66-c83df48263da None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:07.124168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f1a0b-f662-4990-a3be-5e6c5e838564 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:07.124592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f1a0b-f662-4990-a3be-5e6c5e838564 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:07.124941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f1a0b-f662-4990-a3be-5e6c5e838564 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:07.126387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 67/132] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:07.129760 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41a027-f951-4b25-bc66-c83df48263da None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:07.131450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651c592-27b4-4fbe-b2d2-5cd86381963e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:07.131742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651c592-27b4-4fbe-b2d2-5cd86381963e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:07.132026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651c592-27b4-4fbe-b2d2-5cd86381963e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:07.149375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5651c592-27b4-4fbe-b2d2-5cd86381963e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:07.150560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5651c592-27b4-4fbe-b2d2-5cd86381963e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:07.163622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99558d2-4443-4b5a-b27b-e65e5e075c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:07.164257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99558d2-4443-4b5a-b27b-e65e5e075c29 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:07.164570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99558d2-4443-4b5a-b27b-e65e5e075c29 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:07.164851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99558d2-4443-4b5a-b27b-e65e5e075c29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:07.165944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99558d2-4443-4b5a-b27b-e65e5e075c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:07.167376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99558d2-4443-4b5a-b27b-e65e5e075c29 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:07.169464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 66/133] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:07.174880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99558d2-4443-4b5a-b27b-e65e5e075c29 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:07.218186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 68/134] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:07.225102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:07.271704 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99558d2-4443-4b5a-b27b-e65e5e075c29 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:07.304195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaf1c8-73dd-4b56-b381-9026a2882817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:07.304766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaf1c8-73dd-4b56-b381-9026a2882817 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:07.305050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaf1c8-73dd-4b56-b381-9026a2882817 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:07.305326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaf1c8-73dd-4b56-b381-9026a2882817 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:07.305976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaf1c8-73dd-4b56-b381-9026a2882817 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:07.307405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaf1c8-73dd-4b56-b381-9026a2882817 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:07.329138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 67/135] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:07 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:28:07.486414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651c592-27b4-4fbe-b2d2-5cd86381963e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:07.584112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:07.588272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651c592-27b4-4fbe-b2d2-5cd86381963e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:07.625650 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c58a9-3338-4d19-ab7c-09c275eae54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:07.626294 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c58a9-3338-4d19-ab7c-09c275eae54c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:07.626576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c58a9-3338-4d19-ab7c-09c275eae54c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:07.626852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c58a9-3338-4d19-ab7c-09c275eae54c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:07.637820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c58a9-3338-4d19-ab7c-09c275eae54c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-373c58a9-3338-4d19-ab7c-09c275eae54c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:07.642640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:07.643464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:07.644203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 69/136] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:07.649506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaf1c8-73dd-4b56-b381-9026a2882817 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:07.757803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaf1c8-73dd-4b56-b381-9026a2882817 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:07.769590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:07.799551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4a80d-8681-4dff-a66b-40b904aaeddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:07.800169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4a80d-8681-4dff-a66b-40b904aaeddc None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:07.800503 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4a80d-8681-4dff-a66b-40b904aaeddc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:07.800779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4a80d-8681-4dff-a66b-40b904aaeddc None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:07.801443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4a80d-8681-4dff-a66b-40b904aaeddc None admin] RBAC: Adding query filter params (name=reader) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:07.801750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4a80d-8681-4dff-a66b-40b904aaeddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:07.803118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4a80d-8681-4dff-a66b-40b904aaeddc None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:07.812949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 68/137] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:07.818737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c58a9-3338-4d19-ab7c-09c275eae54c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:07.926115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c58a9-3338-4d19-ab7c-09c275eae54c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:07.962642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed78c8-e52e-4e5f-b111-6719e2b9a1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:07.963289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed78c8-e52e-4e5f-b111-6719e2b9a1a1 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:07.963574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed78c8-e52e-4e5f-b111-6719e2b9a1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:07.963856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed78c8-e52e-4e5f-b111-6719e2b9a1a1 None admin] PATH_INFO: `/v3/users/8f9d2f52dec345548775b353df4faccd` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:07.965032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed78c8-e52e-4e5f-b111-6719e2b9a1a1 None admin] RBAC: Authorizing `identity:get_user(user_id=8f9d2f52dec345548775b353df4faccd)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:07.967234 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed78c8-e52e-4e5f-b111-6719e2b9a1a1 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:07.969254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 70/138] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:07 2022] GET /identity/v3/users/8f9d2f52dec345548775b353df4faccd => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:07.974488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4a80d-8681-4dff-a66b-40b904aaeddc None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:08.079140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4a80d-8681-4dff-a66b-40b904aaeddc None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:08.114895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe510a-dd07-4d42-8d78-61e17644f9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:08.115846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe510a-dd07-4d42-8d78-61e17644f9df None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:08.116324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe510a-dd07-4d42-8d78-61e17644f9df None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:08.116840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe510a-dd07-4d42-8d78-61e17644f9df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:08.159675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:08.190809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe510a-dd07-4d42-8d78-61e17644f9df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:08.192622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe510a-dd07-4d42-8d78-61e17644f9df None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:08.292757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 69/139] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:08.299340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed78c8-e52e-4e5f-b111-6719e2b9a1a1 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:08.329511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:08.412187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed78c8-e52e-4e5f-b111-6719e2b9a1a1 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:08.447149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e603126-3197-4c3f-93f4-81e4fa9b0e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:08.447703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e603126-3197-4c3f-93f4-81e4fa9b0e44 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:08.447985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e603126-3197-4c3f-93f4-81e4fa9b0e44 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:08.448262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e603126-3197-4c3f-93f4-81e4fa9b0e44 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:08.449411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e603126-3197-4c3f-93f4-81e4fa9b0e44 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:08.451622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e603126-3197-4c3f-93f4-81e4fa9b0e44 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:08.453561 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 71/140] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:08 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:08.459391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe510a-dd07-4d42-8d78-61e17644f9df None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:08.534874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:08.565873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe510a-dd07-4d42-8d78-61e17644f9df None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:08.602875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c79b3-6e9f-4588-83fd-671858af757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:08.603483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c79b3-6e9f-4588-83fd-671858af757d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:08.603765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c79b3-6e9f-4588-83fd-671858af757d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:08.604038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c79b3-6e9f-4588-83fd-671858af757d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:08.604790 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c79b3-6e9f-4588-83fd-671858af757d None admin] RBAC: Adding query filter params (role.id=c8d552ca72b140a8a815448527770138, scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81, user.id=8f9d2f52dec345548775b353df4faccd) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:08.605099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c79b3-6e9f-4588-83fd-671858af757d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:08.607303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c79b3-6e9f-4588-83fd-671858af757d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:08.613387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 70/141] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:08 2022] GET /identity/v3/role_assignments?user.id=8f9d2f52dec345548775b353df4faccd&scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81&role.id=c8d552ca72b140a8a815448527770138 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:09.627609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6b202-dfcf-43bf-883f-7ed73befc7f9 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:09.628088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6b202-dfcf-43bf-883f-7ed73befc7f9 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:09.628088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6b202-dfcf-43bf-883f-7ed73befc7f9 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:09.629487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 72/142] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:09.634463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a597-486c-4569-b337-1a84c08e1889 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:09.634832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a597-486c-4569-b337-1a84c08e1889 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:09.635013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a597-486c-4569-b337-1a84c08e1889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:09.654410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9272a597-486c-4569-b337-1a84c08e1889 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:09.655753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9272a597-486c-4569-b337-1a84c08e1889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:09.728177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 71/143] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:09.780982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7aa3-a239-4d59-bbfe-467aaf1dd2af None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:09.781294 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7aa3-a239-4d59-bbfe-467aaf1dd2af None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:09.781556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7aa3-a239-4d59-bbfe-467aaf1dd2af None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:09.783119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 73/144] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:09.788893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7853657-7360-4c8c-9168-35142e73b5d9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:09.789181 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7853657-7360-4c8c-9168-35142e73b5d9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:09.789464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7853657-7360-4c8c-9168-35142e73b5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:09.809382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e7853657-7360-4c8c-9168-35142e73b5d9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:09.810758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e7853657-7360-4c8c-9168-35142e73b5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:09.870626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:09.880237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 72/145] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:09.907289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b32f0-5fd9-4433-8c64-38dc5bb5982f None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:09.907570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b32f0-5fd9-4433-8c64-38dc5bb5982f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:09.907847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b32f0-5fd9-4433-8c64-38dc5bb5982f None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:09.909351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 74/146] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:09.914420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c186-593b-4ca1-bea9-091f74b1bae1 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:09.914711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c186-593b-4ca1-bea9-091f74b1bae1 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:09.914989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c186-593b-4ca1-bea9-091f74b1bae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:09.933636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-21e1c186-593b-4ca1-bea9-091f74b1bae1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:09.935494 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-21e1c186-593b-4ca1-bea9-091f74b1bae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:10.001806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 73/147] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:10.004498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b32f0-5fd9-4433-8c64-38dc5bb5982f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.107423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:10.116052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b32f0-5fd9-4433-8c64-38dc5bb5982f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.147021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c186-593b-4ca1-bea9-091f74b1bae1 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.150368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9b061-94ff-4281-9d10-57232f4c7864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.151006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9b061-94ff-4281-9d10-57232f4c7864 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.151291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9b061-94ff-4281-9d10-57232f4c7864 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.151567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9b061-94ff-4281-9d10-57232f4c7864 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.161922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9b061-94ff-4281-9d10-57232f4c7864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-7da9b061-94ff-4281-9d10-57232f4c7864 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:10.166740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:10.167573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:10.169204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 75/148] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:28:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:10.174041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9b061-94ff-4281-9d10-57232f4c7864 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.262411 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c186-593b-4ca1-bea9-091f74b1bae1 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.276647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9b061-94ff-4281-9d10-57232f4c7864 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.287318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:10.298931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2abfda-03c9-4b4d-b51a-e5a4d997fbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.299552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2abfda-03c9-4b4d-b51a-e5a4d997fbc1 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.299827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2abfda-03c9-4b4d-b51a-e5a4d997fbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.300092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2abfda-03c9-4b4d-b51a-e5a4d997fbc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.309905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2abfda-03c9-4b4d-b51a-e5a4d997fbc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:10.311191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ec0cf-37e6-4a86-9a79-0a31837e3a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.311796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ec0cf-37e6-4a86-9a79-0a31837e3a8d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.312078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ec0cf-37e6-4a86-9a79-0a31837e3a8d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.312378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ec0cf-37e6-4a86-9a79-0a31837e3a8d None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.313046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ec0cf-37e6-4a86-9a79-0a31837e3a8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:10.313352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ec0cf-37e6-4a86-9a79-0a31837e3a8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-bb2abfda-03c9-4b4d-b51a-e5a4d997fbc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:10.314652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:10.315508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:10.316254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:10.316947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ec0cf-37e6-4a86-9a79-0a31837e3a8d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:10.317012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 74/149] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:28:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:10.317012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2abfda-03c9-4b4d-b51a-e5a4d997fbc1 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.328427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 76/150] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:10.329478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ec0cf-37e6-4a86-9a79-0a31837e3a8d None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.349113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:10.420324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2abfda-03c9-4b4d-b51a-e5a4d997fbc1 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.431131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ec0cf-37e6-4a86-9a79-0a31837e3a8d None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.454995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac193-b099-4130-a1e2-2d1461f9980f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.455717 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac193-b099-4130-a1e2-2d1461f9980f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.455996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac193-b099-4130-a1e2-2d1461f9980f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.456267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac193-b099-4130-a1e2-2d1461f9980f None admin] PATH_INFO: `/v3/domains/default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.457405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac193-b099-4130-a1e2-2d1461f9980f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:10.458863 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac193-b099-4130-a1e2-2d1461f9980f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:10.460463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 75/151] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:10.461226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac193-b099-4130-a1e2-2d1461f9980f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.468300 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e68b1-d004-46b6-89af-b9d4b2c94195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.468932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e68b1-d004-46b6-89af-b9d4b2c94195 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.469622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e68b1-d004-46b6-89af-b9d4b2c94195 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.469902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e68b1-d004-46b6-89af-b9d4b2c94195 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.470569 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e68b1-d004-46b6-89af-b9d4b2c94195 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:10.470873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e68b1-d004-46b6-89af-b9d4b2c94195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:10.472917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e68b1-d004-46b6-89af-b9d4b2c94195 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:10.480784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 77/152] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:10.482925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e68b1-d004-46b6-89af-b9d4b2c94195 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.512633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:10.558029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac193-b099-4130-a1e2-2d1461f9980f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.562458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:10.583225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e68b1-d004-46b6-89af-b9d4b2c94195 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.591071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d71dcf-25b1-4288-8e50-fac639edadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.591724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d71dcf-25b1-4288-8e50-fac639edadd1 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.592004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d71dcf-25b1-4288-8e50-fac639edadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.592276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d71dcf-25b1-4288-8e50-fac639edadd1 None admin] PATH_INFO: `/v3/users/86084e4889634a69b9cd17a342c972b7` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.607486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d71dcf-25b1-4288-8e50-fac639edadd1 None admin] RBAC: Authorizing `identity:get_user(user_id=86084e4889634a69b9cd17a342c972b7)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:10.608968 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d71dcf-25b1-4288-8e50-fac639edadd1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:10.610584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 76/153] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:10 2022] GET /identity/v3/users/86084e4889634a69b9cd17a342c972b7 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:10.613167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d71dcf-25b1-4288-8e50-fac639edadd1 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.616750 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf2c10-448e-4695-b952-85f0a28bc76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.617360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf2c10-448e-4695-b952-85f0a28bc76e None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.617892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf2c10-448e-4695-b952-85f0a28bc76e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.618161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf2c10-448e-4695-b952-85f0a28bc76e None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.618830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf2c10-448e-4695-b952-85f0a28bc76e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:10.620262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf2c10-448e-4695-b952-85f0a28bc76e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:10.647131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 78/154] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:10 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:10.647938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf2c10-448e-4695-b952-85f0a28bc76e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.731230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d71dcf-25b1-4288-8e50-fac639edadd1 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.744466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:10.762886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf2c10-448e-4695-b952-85f0a28bc76e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.775393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895488f-21d3-441e-b318-01c235852c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.776047 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895488f-21d3-441e-b318-01c235852c04 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.776326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895488f-21d3-441e-b318-01c235852c04 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.776639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895488f-21d3-441e-b318-01c235852c04 None admin] PATH_INFO: `/v3/users/86084e4889634a69b9cd17a342c972b7` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.778647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895488f-21d3-441e-b318-01c235852c04 None admin] RBAC: Authorizing `identity:get_user(user_id=86084e4889634a69b9cd17a342c972b7)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:10.780235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895488f-21d3-441e-b318-01c235852c04 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:10.786609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 77/155] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:10 2022] GET /identity/v3/users/86084e4889634a69b9cd17a342c972b7 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:10.788293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895488f-21d3-441e-b318-01c235852c04 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.802472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fd664-43c1-4980-a4ef-8f1acbfd88b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.803125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fd664-43c1-4980-a4ef-8f1acbfd88b9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.803399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fd664-43c1-4980-a4ef-8f1acbfd88b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.803672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fd664-43c1-4980-a4ef-8f1acbfd88b9 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.805017 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fd664-43c1-4980-a4ef-8f1acbfd88b9 None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:10.806505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fd664-43c1-4980-a4ef-8f1acbfd88b9 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:10.808184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 79/156] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:10 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:10.814144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fd664-43c1-4980-a4ef-8f1acbfd88b9 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.816988 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:10.925525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895488f-21d3-441e-b318-01c235852c04 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.933678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fd664-43c1-4980-a4ef-8f1acbfd88b9 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:10.964242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26679a-ab77-4c9f-abee-d46ceb6fd117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.965812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26679a-ab77-4c9f-abee-d46ceb6fd117 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.965812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26679a-ab77-4c9f-abee-d46ceb6fd117 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.966343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26679a-ab77-4c9f-abee-d46ceb6fd117 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.967371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26679a-ab77-4c9f-abee-d46ceb6fd117 None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:10.968870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26679a-ab77-4c9f-abee-d46ceb6fd117 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:10.970667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 78/157] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:10 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:10.976023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26679a-ab77-4c9f-abee-d46ceb6fd117 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:10.977826 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a239c-5984-4e72-962c-737ca008d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:10.978496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a239c-5984-4e72-962c-737ca008d643 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:10.978766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a239c-5984-4e72-962c-737ca008d643 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:10.979093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a239c-5984-4e72-962c-737ca008d643 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346/users/86084e4889634a69b9cd17a342c972b7/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:10.981156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a239c-5984-4e72-962c-737ca008d643 None admin] RBAC: Authorizing `identity:create_grant(project_id=6da9e929c8864aca8ed43b28bed3d346, user_id=86084e4889634a69b9cd17a342c972b7, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:10.982668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a239c-5984-4e72-962c-737ca008d643 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:10.998841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 80/158] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:10 2022] PUT /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346/users/86084e4889634a69b9cd17a342c972b7/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:28:11.036846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:11.103865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26679a-ab77-4c9f-abee-d46ceb6fd117 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:11.143338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfdaaa-35b3-47be-8269-cb244bf0db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:11.144115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfdaaa-35b3-47be-8269-cb244bf0db41 None admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:11.144525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfdaaa-35b3-47be-8269-cb244bf0db41 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:11.144864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfdaaa-35b3-47be-8269-cb244bf0db41 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346/users/86084e4889634a69b9cd17a342c972b7/roles/dff349185bc94cb4a0e0f0c40425862f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:11.147186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdfdaaa-35b3-47be-8269-cb244bf0db41 None admin] RBAC: Authorizing `identity:create_grant(project_id=6da9e929c8864aca8ed43b28bed3d346, user_id=86084e4889634a69b9cd17a342c972b7, role_id=dff349185bc94cb4a0e0f0c40425862f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:11.148853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdfdaaa-35b3-47be-8269-cb244bf0db41 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:11.165316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 79/159] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:10 2022] PUT /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346/users/86084e4889634a69b9cd17a342c972b7/roles/dff349185bc94cb4a0e0f0c40425862f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:28:12.986783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e0de4-fe85-44ef-b108-d9615bee737d None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:12.987337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e0de4-fe85-44ef-b108-d9615bee737d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:12.987337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e0de4-fe85-44ef-b108-d9615bee737d None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:12.988701 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 81/160] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:12.994144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2be694-40dc-4b49-9956-ca6bdec08828 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:12.994551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2be694-40dc-4b49-9956-ca6bdec08828 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:12.994694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2be694-40dc-4b49-9956-ca6bdec08828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.013321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0c2be694-40dc-4b49-9956-ca6bdec08828 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:13.014579 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0c2be694-40dc-4b49-9956-ca6bdec08828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:13.088429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 80/161] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:13.166309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03726c-e609-4ab7-8dee-6a4bab11fddb None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.166633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03726c-e609-4ab7-8dee-6a4bab11fddb None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.166874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03726c-e609-4ab7-8dee-6a4bab11fddb None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.168916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 82/162] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:13.176219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e5852-a8cf-4171-a9fd-2559e3cac3ab None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.176714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e5852-a8cf-4171-a9fd-2559e3cac3ab None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.177153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e5852-a8cf-4171-a9fd-2559e3cac3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.195892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1f8e5852-a8cf-4171-a9fd-2559e3cac3ab None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:13.197383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1f8e5852-a8cf-4171-a9fd-2559e3cac3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:13.199252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c6524-b64e-420b-aa66-d2a95aa8e64f None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.200068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c6524-b64e-420b-aa66-d2a95aa8e64f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.200376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c6524-b64e-420b-aa66-d2a95aa8e64f None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.202110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 83/163] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:28:13.207689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340ac89-d406-4179-a71a-8a7e1a210e06 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.207972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340ac89-d406-4179-a71a-8a7e1a210e06 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.208249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340ac89-d406-4179-a71a-8a7e1a210e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.228509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-a340ac89-d406-4179-a71a-8a7e1a210e06 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:13.229713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a340ac89-d406-4179-a71a-8a7e1a210e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:13.238910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:13.293597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:13.299230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 81/164] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:13.319671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 84/165] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:13.323709 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4119b6-99e2-431d-a285-5200c7c186e3 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.323998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4119b6-99e2-431d-a285-5200c7c186e3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.324275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4119b6-99e2-431d-a285-5200c7c186e3 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.325821 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 82/166] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:13.330814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264512d-4690-42e4-9369-1144fd60e73a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.331265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264512d-4690-42e4-9369-1144fd60e73a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.331539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264512d-4690-42e4-9369-1144fd60e73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.356895 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b264512d-4690-42e4-9369-1144fd60e73a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:13.356895 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b264512d-4690-42e4-9369-1144fd60e73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:13.377323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4119b6-99e2-431d-a285-5200c7c186e3 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:13.444724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 85/167] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:13.484769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4119b6-99e2-431d-a285-5200c7c186e3 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:13.521264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75d3e3-e5aa-4685-97cb-21461c11e0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:13.521830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75d3e3-e5aa-4685-97cb-21461c11e0d6 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.522107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75d3e3-e5aa-4685-97cb-21461c11e0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.522381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75d3e3-e5aa-4685-97cb-21461c11e0d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.533722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75d3e3-e5aa-4685-97cb-21461c11e0d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-0b75d3e3-e5aa-4685-97cb-21461c11e0d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:13.538531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:13.539418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:13.540297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 83/168] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:13.549424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264512d-4690-42e4-9369-1144fd60e73a None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:13.596536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75d3e3-e5aa-4685-97cb-21461c11e0d6 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:13.626474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:13.656199 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264512d-4690-42e4-9369-1144fd60e73a None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:13.682815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:13.696207 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec6c82-e8e9-4931-a3b3-5a6d0defae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:13.696866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec6c82-e8e9-4931-a3b3-5a6d0defae47 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.697145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec6c82-e8e9-4931-a3b3-5a6d0defae47 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.697423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec6c82-e8e9-4931-a3b3-5a6d0defae47 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.698096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec6c82-e8e9-4931-a3b3-5a6d0defae47 None admin] RBAC: Adding query filter params (name=member) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:13.698403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec6c82-e8e9-4931-a3b3-5a6d0defae47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:13.700438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec6c82-e8e9-4931-a3b3-5a6d0defae47 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:13.704557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75d3e3-e5aa-4685-97cb-21461c11e0d6 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:13.708861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 86/169] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:13.709487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec6c82-e8e9-4931-a3b3-5a6d0defae47 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:13.744143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11060c7b-6f41-4a96-a609-dff01d076b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:13.744794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11060c7b-6f41-4a96-a609-dff01d076b49 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.745069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11060c7b-6f41-4a96-a609-dff01d076b49 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.745337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11060c7b-6f41-4a96-a609-dff01d076b49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.756352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11060c7b-6f41-4a96-a609-dff01d076b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-11060c7b-6f41-4a96-a609-dff01d076b49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:13.761551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:13.762559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:13.763298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 84/170] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:28:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:13.763977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11060c7b-6f41-4a96-a609-dff01d076b49 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:13.815633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec6c82-e8e9-4931-a3b3-5a6d0defae47 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:13.847339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:13.847887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464903-511b-4b75-9d6f-fef414f5b814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:13.848619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464903-511b-4b75-9d6f-fef414f5b814 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.848916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464903-511b-4b75-9d6f-fef414f5b814 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.849233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464903-511b-4b75-9d6f-fef414f5b814 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.850413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86464903-511b-4b75-9d6f-fef414f5b814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:13.851863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86464903-511b-4b75-9d6f-fef414f5b814 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:13.853471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 87/171] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:13.859276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464903-511b-4b75-9d6f-fef414f5b814 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:13.862420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11060c7b-6f41-4a96-a609-dff01d076b49 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:13.897442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602251d-2f19-485a-ab3e-b75a5c01fe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:13.898072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602251d-2f19-485a-ab3e-b75a5c01fe92 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.898353 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602251d-2f19-485a-ab3e-b75a5c01fe92 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.898629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602251d-2f19-485a-ab3e-b75a5c01fe92 None admin] PATH_INFO: `/v3/users/b3da93308bc647c3ac068addb4beb043` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:13.911067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:13.912372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602251d-2f19-485a-ab3e-b75a5c01fe92 None admin] RBAC: Authorizing `identity:get_user(user_id=b3da93308bc647c3ac068addb4beb043)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:13.914050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602251d-2f19-485a-ab3e-b75a5c01fe92 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:13.915735 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 85/172] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:13 2022] GET /identity/v3/users/b3da93308bc647c3ac068addb4beb043 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:13.921147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602251d-2f19-485a-ab3e-b75a5c01fe92 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:13.964785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464903-511b-4b75-9d6f-fef414f5b814 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:13.997734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce3ea5-6f12-40e1-8c9d-5ca9170b5ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:13.998350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce3ea5-6f12-40e1-8c9d-5ca9170b5ad6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:13.998614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce3ea5-6f12-40e1-8c9d-5ca9170b5ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:13.998880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce3ea5-6f12-40e1-8c9d-5ca9170b5ad6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.008583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce3ea5-6f12-40e1-8c9d-5ca9170b5ad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-d4ce3ea5-6f12-40e1-8c9d-5ca9170b5ad6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:14.013838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:14.014696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:14.015492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 88/173] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:28:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:14.016132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce3ea5-6f12-40e1-8c9d-5ca9170b5ad6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:14.022381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602251d-2f19-485a-ab3e-b75a5c01fe92 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:14.056587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19111d-c9c2-4b31-8c27-074b13d4446a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:14.057192 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19111d-c9c2-4b31-8c27-074b13d4446a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:14.057471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19111d-c9c2-4b31-8c27-074b13d4446a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:14.057743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19111d-c9c2-4b31-8c27-074b13d4446a None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.058398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19111d-c9c2-4b31-8c27-074b13d4446a None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:14.058700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19111d-c9c2-4b31-8c27-074b13d4446a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:14.060112 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19111d-c9c2-4b31-8c27-074b13d4446a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:14.067913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 86/174] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:14.068668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19111d-c9c2-4b31-8c27-074b13d4446a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:14.077053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:14.114456 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce3ea5-6f12-40e1-8c9d-5ca9170b5ad6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:14.127703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:14.148387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a222-953e-41ae-9312-53694943705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:14.148989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a222-953e-41ae-9312-53694943705c None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:14.149267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a222-953e-41ae-9312-53694943705c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:14.149542 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a222-953e-41ae-9312-53694943705c None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.150193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7a222-953e-41ae-9312-53694943705c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:14.151646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7a222-953e-41ae-9312-53694943705c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:14.168824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19111d-c9c2-4b31-8c27-074b13d4446a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:14.173267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 89/175] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:14 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:14.174075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a222-953e-41ae-9312-53694943705c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:14.217419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c59ec-aa84-4867-b4fa-c38254f1c6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:14.218137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c59ec-aa84-4867-b4fa-c38254f1c6b1 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:14.218419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c59ec-aa84-4867-b4fa-c38254f1c6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:14.218708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c59ec-aa84-4867-b4fa-c38254f1c6b1 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.219889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c59ec-aa84-4867-b4fa-c38254f1c6b1 None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:14.221948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c59ec-aa84-4867-b4fa-c38254f1c6b1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:14.229593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 87/176] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:14 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:14.229593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c59ec-aa84-4867-b4fa-c38254f1c6b1 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:14.290438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a222-953e-41ae-9312-53694943705c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:14.317575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:14.333909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774725e2-85ca-40e3-ac74-e30977a27dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:14.334440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774725e2-85ca-40e3-ac74-e30977a27dc2 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:14.334824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774725e2-85ca-40e3-ac74-e30977a27dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:14.335095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774725e2-85ca-40e3-ac74-e30977a27dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.335881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774725e2-85ca-40e3-ac74-e30977a27dc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:14.336183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774725e2-85ca-40e3-ac74-e30977a27dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:14.339367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774725e2-85ca-40e3-ac74-e30977a27dc2 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:14.349167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 90/177] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:14.350059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774725e2-85ca-40e3-ac74-e30977a27dc2 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:14.356451 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c59ec-aa84-4867-b4fa-c38254f1c6b1 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:14.387737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:14.398725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781df9-1ab0-4900-9c04-96397ce41ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:14.400225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781df9-1ab0-4900-9c04-96397ce41ae3 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:14.400225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781df9-1ab0-4900-9c04-96397ce41ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:14.400225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781df9-1ab0-4900-9c04-96397ce41ae3 None admin] PATH_INFO: `/v3/users/86084e4889634a69b9cd17a342c972b7` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.400959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00781df9-1ab0-4900-9c04-96397ce41ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=86084e4889634a69b9cd17a342c972b7)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:14.402410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00781df9-1ab0-4900-9c04-96397ce41ae3 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:14.405557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 88/178] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:14 2022] GET /identity/v3/users/86084e4889634a69b9cd17a342c972b7 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:14.405557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781df9-1ab0-4900-9c04-96397ce41ae3 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:14.473405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774725e2-85ca-40e3-ac74-e30977a27dc2 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:14.511800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37b71f-332e-4108-9a35-984f6e0cbbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:14.513343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37b71f-332e-4108-9a35-984f6e0cbbb7 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:14.513624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37b71f-332e-4108-9a35-984f6e0cbbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:14.513899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37b71f-332e-4108-9a35-984f6e0cbbb7 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346/users/b3da93308bc647c3ac068addb4beb043/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.515893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37b71f-332e-4108-9a35-984f6e0cbbb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6da9e929c8864aca8ed43b28bed3d346, user_id=b3da93308bc647c3ac068addb4beb043, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:14.518047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37b71f-332e-4108-9a35-984f6e0cbbb7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:14.533534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781df9-1ab0-4900-9c04-96397ce41ae3 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:14.533864 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 91/179] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:14 2022] PUT /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346/users/b3da93308bc647c3ac068addb4beb043/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:14.534915 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37b71f-332e-4108-9a35-984f6e0cbbb7 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:14.587160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c753b-7450-4102-9464-c3bd4746492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:14.588095 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c753b-7450-4102-9464-c3bd4746492a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:14.588866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c753b-7450-4102-9464-c3bd4746492a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:14.589342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c753b-7450-4102-9464-c3bd4746492a None admin] PATH_INFO: `/v3/users/86084e4889634a69b9cd17a342c972b7` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.591918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c753b-7450-4102-9464-c3bd4746492a None admin] RBAC: Authorizing `identity:get_user(user_id=86084e4889634a69b9cd17a342c972b7)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:14.593725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c753b-7450-4102-9464-c3bd4746492a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:14.595647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 89/180] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:14 2022] GET /identity/v3/users/86084e4889634a69b9cd17a342c972b7 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:14.603122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c753b-7450-4102-9464-c3bd4746492a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:14.622799 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:14.674583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:14.689254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37b71f-332e-4108-9a35-984f6e0cbbb7 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:14.735005 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529899a6-95f9-4343-8365-e032e82ec80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:14.735637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529899a6-95f9-4343-8365-e032e82ec80c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:14.735917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529899a6-95f9-4343-8365-e032e82ec80c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:14.736194 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529899a6-95f9-4343-8365-e032e82ec80c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.743222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c753b-7450-4102-9464-c3bd4746492a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:14.792338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5795f8-b67a-4fa1-95f4-e6e504ab3d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:14.793191 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5795f8-b67a-4fa1-95f4-e6e504ab3d64 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:14.793594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5795f8-b67a-4fa1-95f4-e6e504ab3d64 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:14.793872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5795f8-b67a-4fa1-95f4-e6e504ab3d64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:14.830874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529899a6-95f9-4343-8365-e032e82ec80c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:14.833688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529899a6-95f9-4343-8365-e032e82ec80c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:14.884634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5795f8-b67a-4fa1-95f4-e6e504ab3d64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:14.886170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5795f8-b67a-4fa1-95f4-e6e504ab3d64 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:14.901950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:14.944152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:14.956378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 92/181] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:14.969367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529899a6-95f9-4343-8365-e032e82ec80c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:15.019559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 90/182] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:15.029020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5795f8-b67a-4fa1-95f4-e6e504ab3d64 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:15.102284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529899a6-95f9-4343-8365-e032e82ec80c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:15.141918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154e846-0e9b-4c79-9895-446ef802ff67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:15.142906 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154e846-0e9b-4c79-9895-446ef802ff67 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:15.143389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154e846-0e9b-4c79-9895-446ef802ff67 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:15.143883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154e846-0e9b-4c79-9895-446ef802ff67 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:15.145327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154e846-0e9b-4c79-9895-446ef802ff67 None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:15.147008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154e846-0e9b-4c79-9895-446ef802ff67 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:15.148885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 93/183] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:14 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:15.156366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154e846-0e9b-4c79-9895-446ef802ff67 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:15.203986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5795f8-b67a-4fa1-95f4-e6e504ab3d64 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:15.216782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:15.255514 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03daa6-fa55-4256-817b-89b5b370f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:15.256149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03daa6-fa55-4256-817b-89b5b370f4ae None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:15.256501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03daa6-fa55-4256-817b-89b5b370f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:15.256975 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03daa6-fa55-4256-817b-89b5b370f4ae None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:15.258816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03daa6-fa55-4256-817b-89b5b370f4ae None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:15.261002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03daa6-fa55-4256-817b-89b5b370f4ae None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:15.263739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 91/184] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:15 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:15.266162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:15.269614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03daa6-fa55-4256-817b-89b5b370f4ae None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:15.346795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154e846-0e9b-4c79-9895-446ef802ff67 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:15.413641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de2d94-154a-4976-8597-6a5aa13a5676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:15.414171 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de2d94-154a-4976-8597-6a5aa13a5676 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:15.414447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de2d94-154a-4976-8597-6a5aa13a5676 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:15.414717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de2d94-154a-4976-8597-6a5aa13a5676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:15.415445 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de2d94-154a-4976-8597-6a5aa13a5676 None admin] RBAC: Adding query filter params (role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.project.id=6da9e929c8864aca8ed43b28bed3d346, user.id=86084e4889634a69b9cd17a342c972b7) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:15.415748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de2d94-154a-4976-8597-6a5aa13a5676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:15.417209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de2d94-154a-4976-8597-6a5aa13a5676 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:15.430315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 94/185] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:15 2022] GET /identity/v3/role_assignments?user.id=86084e4889634a69b9cd17a342c972b7&scope.project.id=6da9e929c8864aca8ed43b28bed3d346&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:15.446426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03daa6-fa55-4256-817b-89b5b370f4ae None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:15.509464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60a64f-920a-4a8e-93f7-b89000f6ac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:15.510088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60a64f-920a-4a8e-93f7-b89000f6ac52 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:15.510362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60a64f-920a-4a8e-93f7-b89000f6ac52 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:15.510633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60a64f-920a-4a8e-93f7-b89000f6ac52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:15.511366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60a64f-920a-4a8e-93f7-b89000f6ac52 None admin] RBAC: Adding query filter params (role.id=dff349185bc94cb4a0e0f0c40425862f, scope.project.id=6da9e929c8864aca8ed43b28bed3d346, user.id=86084e4889634a69b9cd17a342c972b7) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:15.511671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60a64f-920a-4a8e-93f7-b89000f6ac52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:15.513397 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60a64f-920a-4a8e-93f7-b89000f6ac52 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:15.522210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 92/186] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:15 2022] GET /identity/v3/role_assignments?user.id=86084e4889634a69b9cd17a342c972b7&scope.project.id=6da9e929c8864aca8ed43b28bed3d346&role.id=dff349185bc94cb4a0e0f0c40425862f => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:16.545060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac3ec3-f858-4fcd-b1ff-8f6777c0726c None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:16.545590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac3ec3-f858-4fcd-b1ff-8f6777c0726c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:16.545590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac3ec3-f858-4fcd-b1ff-8f6777c0726c None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:16.547106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 95/187] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:16.552119 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115ac6b-8c46-49dd-96e1-20683fa0fddc None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:16.552528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115ac6b-8c46-49dd-96e1-20683fa0fddc None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:16.552722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115ac6b-8c46-49dd-96e1-20683fa0fddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:16.571112 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e115ac6b-8c46-49dd-96e1-20683fa0fddc None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:16.572303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e115ac6b-8c46-49dd-96e1-20683fa0fddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:16.609943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:16.643462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 93/188] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:16.912499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac3ec3-f858-4fcd-b1ff-8f6777c0726c None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:16.951222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24453378-bb94-481f-b2d0-0f41e11c104b None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:16.951543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24453378-bb94-481f-b2d0-0f41e11c104b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:16.951769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24453378-bb94-481f-b2d0-0f41e11c104b None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:16.953140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 94/189] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:28:16.958348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b11ba-c6f0-4a4d-b36d-6069d15f7647 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:16.958704 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b11ba-c6f0-4a4d-b36d-6069d15f7647 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:16.959051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b11ba-c6f0-4a4d-b36d-6069d15f7647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:16.979246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-549b11ba-c6f0-4a4d-b36d-6069d15f7647 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:16.980603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-549b11ba-c6f0-4a4d-b36d-6069d15f7647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:17.020177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac3ec3-f858-4fcd-b1ff-8f6777c0726c None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:17.041720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:17.048239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 95/190] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:17.049875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa89fd-f04e-4e49-9768-466c11b774a0 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.050170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa89fd-f04e-4e49-9768-466c11b774a0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.050449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa89fd-f04e-4e49-9768-466c11b774a0 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.051804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 96/191] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:17.055884 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac52c9-8213-4ec9-bd8b-fe5a423636bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:17.056557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac52c9-8213-4ec9-bd8b-fe5a423636bd None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.056842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac52c9-8213-4ec9-bd8b-fe5a423636bd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.057006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb174a-00c7-4be3-8a47-aae1d2892de6 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.057101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac52c9-8213-4ec9-bd8b-fe5a423636bd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.057280 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb174a-00c7-4be3-8a47-aae1d2892de6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.057564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb174a-00c7-4be3-8a47-aae1d2892de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.068142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac52c9-8213-4ec9-bd8b-fe5a423636bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-daac52c9-8213-4ec9-bd8b-fe5a423636bd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:17.073240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:17.074273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:17.075400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:17.076134 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0ceb174a-00c7-4be3-8a47-aae1d2892de6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:17.077210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0ceb174a-00c7-4be3-8a47-aae1d2892de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:17.077593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 96/192] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:17.083183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac52c9-8213-4ec9-bd8b-fe5a423636bd None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:17.153806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 97/193] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:17.184560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac52c9-8213-4ec9-bd8b-fe5a423636bd None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:17.218768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ee2bf-ff51-4301-b34d-040d3f479a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:17.219582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ee2bf-ff51-4301-b34d-040d3f479a54 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.220072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ee2bf-ff51-4301-b34d-040d3f479a54 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.220523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ee2bf-ff51-4301-b34d-040d3f479a54 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.221341 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ee2bf-ff51-4301-b34d-040d3f479a54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:17.221789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ee2bf-ff51-4301-b34d-040d3f479a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:17.223335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ee2bf-ff51-4301-b34d-040d3f479a54 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:17.231377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 97/194] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:17.236920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb174a-00c7-4be3-8a47-aae1d2892de6 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:17.271088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:17.317265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ee2bf-ff51-4301-b34d-040d3f479a54 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:17.343727 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb174a-00c7-4be3-8a47-aae1d2892de6 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:17.370511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:17.381026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7359fe-b18e-45a2-b5ec-b3ebf87a6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:17.381801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7359fe-b18e-45a2-b5ec-b3ebf87a6c8d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.382090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7359fe-b18e-45a2-b5ec-b3ebf87a6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.382509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7359fe-b18e-45a2-b5ec-b3ebf87a6c8d None admin] PATH_INFO: `/v3/users/503642c6603642ef9f94a00fb1aa4a2d` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.396564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7359fe-b18e-45a2-b5ec-b3ebf87a6c8d None admin] RBAC: Authorizing `identity:get_user(user_id=503642c6603642ef9f94a00fb1aa4a2d)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:17.398033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7359fe-b18e-45a2-b5ec-b3ebf87a6c8d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:17.399655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 98/195] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:17 2022] GET /identity/v3/users/503642c6603642ef9f94a00fb1aa4a2d => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:17.405206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7359fe-b18e-45a2-b5ec-b3ebf87a6c8d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:17.432316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ee2bf-ff51-4301-b34d-040d3f479a54 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:17.466312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63602-5a35-47cf-a5fa-62637941bd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:17.466944 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63602-5a35-47cf-a5fa-62637941bd6d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.467229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63602-5a35-47cf-a5fa-62637941bd6d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.467506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63602-5a35-47cf-a5fa-62637941bd6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.468640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f63602-5a35-47cf-a5fa-62637941bd6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:17.470099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f63602-5a35-47cf-a5fa-62637941bd6d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:17.471669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 98/196] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:17.472450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63602-5a35-47cf-a5fa-62637941bd6d None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:17.495644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:17.505515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7359fe-b18e-45a2-b5ec-b3ebf87a6c8d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:17.538971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2c42f-a029-4930-8a47-5fda6da036b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:17.539595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2c42f-a029-4930-8a47-5fda6da036b6 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.539870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2c42f-a029-4930-8a47-5fda6da036b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.540139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2c42f-a029-4930-8a47-5fda6da036b6 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.541302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2c42f-a029-4930-8a47-5fda6da036b6 None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:17.542743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2c42f-a029-4930-8a47-5fda6da036b6 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:17.544437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 99/197] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:17 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:17.545181 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2c42f-a029-4930-8a47-5fda6da036b6 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:17.572067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63602-5a35-47cf-a5fa-62637941bd6d None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:17.585084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:17.606255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21c11b-ba90-4008-b5d0-f3a7e0cb5708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:17.606875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21c11b-ba90-4008-b5d0-f3a7e0cb5708 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.607149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21c11b-ba90-4008-b5d0-f3a7e0cb5708 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.607417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21c11b-ba90-4008-b5d0-f3a7e0cb5708 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.617271 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21c11b-ba90-4008-b5d0-f3a7e0cb5708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-ab21c11b-ba90-4008-b5d0-f3a7e0cb5708 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:17.623322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:17.624338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 99/198] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:17.625329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21c11b-ba90-4008-b5d0-f3a7e0cb5708 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:17.644860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2c42f-a029-4930-8a47-5fda6da036b6 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:17.677844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdbd7b-ccc9-498a-9776-630d46e7e9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:17.678450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdbd7b-ccc9-498a-9776-630d46e7e9fd None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.678728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdbd7b-ccc9-498a-9776-630d46e7e9fd None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.679000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdbd7b-ccc9-498a-9776-630d46e7e9fd None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.679654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdbd7b-ccc9-498a-9776-630d46e7e9fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:17.681113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdbd7b-ccc9-498a-9776-630d46e7e9fd None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:17.702634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 100/199] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:17 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:17.703435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdbd7b-ccc9-498a-9776-630d46e7e9fd None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:17.724127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:17.732409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21c11b-ba90-4008-b5d0-f3a7e0cb5708 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:17.773016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d399-5ee1-41eb-b40c-28327c70581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:17.773926 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d399-5ee1-41eb-b40c-28327c70581e None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.774562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d399-5ee1-41eb-b40c-28327c70581e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.774846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d399-5ee1-41eb-b40c-28327c70581e None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346/users/503642c6603642ef9f94a00fb1aa4a2d/roles/c8d552ca72b140a8a815448527770138` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.778033 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d399-5ee1-41eb-b40c-28327c70581e None admin] RBAC: Authorizing `identity:create_grant(project_id=6da9e929c8864aca8ed43b28bed3d346, user_id=503642c6603642ef9f94a00fb1aa4a2d, role_id=c8d552ca72b140a8a815448527770138)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:17.779891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d399-5ee1-41eb-b40c-28327c70581e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:17.808372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 100/200] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:17 2022] PUT /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346/users/503642c6603642ef9f94a00fb1aa4a2d/roles/c8d552ca72b140a8a815448527770138 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:17.832524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdbd7b-ccc9-498a-9776-630d46e7e9fd None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:17.872881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159d8c2-d666-4303-95c7-45d636fdb16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:17.873545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159d8c2-d666-4303-95c7-45d636fdb16e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:17.873829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159d8c2-d666-4303-95c7-45d636fdb16e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:17.874102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159d8c2-d666-4303-95c7-45d636fdb16e None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:17.874786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159d8c2-d666-4303-95c7-45d636fdb16e None admin] RBAC: Adding query filter params (name=member) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:17.875093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159d8c2-d666-4303-95c7-45d636fdb16e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:17.876559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159d8c2-d666-4303-95c7-45d636fdb16e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:17.892791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 101/201] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:17.894423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d399-5ee1-41eb-b40c-28327c70581e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:17.933315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:18.012980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d399-5ee1-41eb-b40c-28327c70581e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:18.048954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02461d7-0d7b-4d58-8062-e58914f06cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:18.049593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02461d7-0d7b-4d58-8062-e58914f06cb6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:18.049873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02461d7-0d7b-4d58-8062-e58914f06cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:18.050142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02461d7-0d7b-4d58-8062-e58914f06cb6 None admin] PATH_INFO: `/v3/users/b3da93308bc647c3ac068addb4beb043` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:18.051280 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02461d7-0d7b-4d58-8062-e58914f06cb6 None admin] RBAC: Authorizing `identity:get_user(user_id=b3da93308bc647c3ac068addb4beb043)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:18.052780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02461d7-0d7b-4d58-8062-e58914f06cb6 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:18.054527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 101/202] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:17 2022] GET /identity/v3/users/b3da93308bc647c3ac068addb4beb043 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:18.060826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159d8c2-d666-4303-95c7-45d636fdb16e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:18.153590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:18.172003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159d8c2-d666-4303-95c7-45d636fdb16e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:18.206557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e626b-6e37-469d-bc14-a04fde59f1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:18.207165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e626b-6e37-469d-bc14-a04fde59f1c5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:18.207442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e626b-6e37-469d-bc14-a04fde59f1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:18.207715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e626b-6e37-469d-bc14-a04fde59f1c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:18.277316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e626b-6e37-469d-bc14-a04fde59f1c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:18.279115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e626b-6e37-469d-bc14-a04fde59f1c5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:18.377796 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:18.392047 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 102/203] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 20 14:28:18.398320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02461d7-0d7b-4d58-8062-e58914f06cb6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:18.497601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:18.500975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02461d7-0d7b-4d58-8062-e58914f06cb6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:18.536904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd5258-13e5-4e0b-bcf1-a2c4e72db4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:18.537861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd5258-13e5-4e0b-bcf1-a2c4e72db4dd None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:18.538338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd5258-13e5-4e0b-bcf1-a2c4e72db4dd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:18.538816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd5258-13e5-4e0b-bcf1-a2c4e72db4dd None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:18.540162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd5258-13e5-4e0b-bcf1-a2c4e72db4dd None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:18.541854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd5258-13e5-4e0b-bcf1-a2c4e72db4dd None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:18.544393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 102/204] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:18 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:18.550566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e626b-6e37-469d-bc14-a04fde59f1c5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:18.658909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e626b-6e37-469d-bc14-a04fde59f1c5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:18.694856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8911e7-d7be-42e9-a616-b2c48fbf50c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:18.695468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8911e7-d7be-42e9-a616-b2c48fbf50c2 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:18.695749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8911e7-d7be-42e9-a616-b2c48fbf50c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:18.696020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8911e7-d7be-42e9-a616-b2c48fbf50c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:18.696783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8911e7-d7be-42e9-a616-b2c48fbf50c2 None admin] RBAC: Adding query filter params (role.id=a73215dcd05645c89a2f742200bbb0c9, scope.project.id=6da9e929c8864aca8ed43b28bed3d346, user.id=b3da93308bc647c3ac068addb4beb043) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:18.697094 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8911e7-d7be-42e9-a616-b2c48fbf50c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:18.698542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8911e7-d7be-42e9-a616-b2c48fbf50c2 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:18.704972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 103/205] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:18 2022] GET /identity/v3/role_assignments?user.id=b3da93308bc647c3ac068addb4beb043&scope.project.id=6da9e929c8864aca8ed43b28bed3d346&role.id=a73215dcd05645c89a2f742200bbb0c9 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:20.017426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f031594-a53a-4e5a-ada1-5e042d122afe None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.017941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f031594-a53a-4e5a-ada1-5e042d122afe None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.017941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f031594-a53a-4e5a-ada1-5e042d122afe None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.019329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 103/206] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:20.024572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933a221-1d53-4e14-af82-d160a19da4f7 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.024940 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933a221-1d53-4e14-af82-d160a19da4f7 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.025129 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933a221-1d53-4e14-af82-d160a19da4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.042823 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-7933a221-1d53-4e14-af82-d160a19da4f7 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:20.044050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7933a221-1d53-4e14-af82-d160a19da4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:20.083391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:20.097321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ad47-d8e6-4ff8-9b66-104489a35658 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.097628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ad47-d8e6-4ff8-9b66-104489a35658 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.097866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ad47-d8e6-4ff8-9b66-104489a35658 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.099423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 104/207] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:20.104151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9905ad-013d-4a74-85ee-6b57a95b33db None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.104471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9905ad-013d-4a74-85ee-6b57a95b33db None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.104747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9905ad-013d-4a74-85ee-6b57a95b33db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.123015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 104/208] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:20.124395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-8d9905ad-013d-4a74-85ee-6b57a95b33db None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:20.125584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8d9905ad-013d-4a74-85ee-6b57a95b33db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:20.177755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e020a9-2c70-4477-b772-b31db83d798b None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.178048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e020a9-2c70-4477-b772-b31db83d798b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.178321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e020a9-2c70-4477-b772-b31db83d798b None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.179783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 105/209] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:28:20.185016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd7b7-4589-42a3-9d5c-4e756bda0e48 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.185298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd7b7-4589-42a3-9d5c-4e756bda0e48 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.185573 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd7b7-4589-42a3-9d5c-4e756bda0e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.196380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 105/210] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:20.206095 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d0fbd7b7-4589-42a3-9d5c-4e756bda0e48 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:20.207272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d0fbd7b7-4589-42a3-9d5c-4e756bda0e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:20.275987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 106/211] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:20.395226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9905ad-013d-4a74-85ee-6b57a95b33db None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:20.466748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbd7b7-4589-42a3-9d5c-4e756bda0e48 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:20.493508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:20.497625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9905ad-013d-4a74-85ee-6b57a95b33db None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:20.533103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad077c41-737f-47bf-9b56-cafc4edba487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:20.533655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad077c41-737f-47bf-9b56-cafc4edba487 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.533932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad077c41-737f-47bf-9b56-cafc4edba487 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.534203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad077c41-737f-47bf-9b56-cafc4edba487 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.545780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad077c41-737f-47bf-9b56-cafc4edba487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-ad077c41-737f-47bf-9b56-cafc4edba487 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:20.550592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:20.551539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:20.552287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:20.556410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 106/212] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:20.557248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad077c41-737f-47bf-9b56-cafc4edba487 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:20.571225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:20.574521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbd7b7-4589-42a3-9d5c-4e756bda0e48 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:20.608690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f8146-1785-4433-8961-29520094e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:20.609317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f8146-1785-4433-8961-29520094e1ab None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.609591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f8146-1785-4433-8961-29520094e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.609857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f8146-1785-4433-8961-29520094e1ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.619785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f8146-1785-4433-8961-29520094e1ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-215f8146-1785-4433-8961-29520094e1ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:20.625435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:20.626519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 107/213] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:20.627479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f8146-1785-4433-8961-29520094e1ab None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:20.659146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad077c41-737f-47bf-9b56-cafc4edba487 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:20.693430 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25816b-b4a1-4b58-8eac-5ce8b83ad9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:20.694063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25816b-b4a1-4b58-8eac-5ce8b83ad9e2 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.694397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25816b-b4a1-4b58-8eac-5ce8b83ad9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.694674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25816b-b4a1-4b58-8eac-5ce8b83ad9e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.695778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25816b-b4a1-4b58-8eac-5ce8b83ad9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:20.697272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25816b-b4a1-4b58-8eac-5ce8b83ad9e2 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:20.698837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 107/214] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:20.699627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25816b-b4a1-4b58-8eac-5ce8b83ad9e2 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:20.728617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:20.728785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f8146-1785-4433-8961-29520094e1ab None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:20.764577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634fe4-f466-4050-a55c-67e62df4bee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:20.765181 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634fe4-f466-4050-a55c-67e62df4bee4 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.765457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634fe4-f466-4050-a55c-67e62df4bee4 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.765729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634fe4-f466-4050-a55c-67e62df4bee4 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.766396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16634fe4-f466-4050-a55c-67e62df4bee4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:20.766699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16634fe4-f466-4050-a55c-67e62df4bee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:20.768077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16634fe4-f466-4050-a55c-67e62df4bee4 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:20.775836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 108/215] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:20.776645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634fe4-f466-4050-a55c-67e62df4bee4 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:20.799574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:20.806857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25816b-b4a1-4b58-8eac-5ce8b83ad9e2 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:20.840446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b794e7a-249b-471f-a37f-148441d9d429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:20.841197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b794e7a-249b-471f-a37f-148441d9d429 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.841471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b794e7a-249b-471f-a37f-148441d9d429 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.841736 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b794e7a-249b-471f-a37f-148441d9d429 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.842391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b794e7a-249b-471f-a37f-148441d9d429 None admin] RBAC: Adding query filter params (name=member) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:20.842689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b794e7a-249b-471f-a37f-148441d9d429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:20.844050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b794e7a-249b-471f-a37f-148441d9d429 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:20.852526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 108/216] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:20.852526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b794e7a-249b-471f-a37f-148441d9d429 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:20.874886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634fe4-f466-4050-a55c-67e62df4bee4 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:20.909442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b03c5-0153-4953-9e9f-63ada6ea7f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:20.910049 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b03c5-0153-4953-9e9f-63ada6ea7f1f None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:20.910325 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b03c5-0153-4953-9e9f-63ada6ea7f1f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:20.910595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b03c5-0153-4953-9e9f-63ada6ea7f1f None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:20.911263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b03c5-0153-4953-9e9f-63ada6ea7f1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:20.912714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b03c5-0153-4953-9e9f-63ada6ea7f1f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:20.948376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 109/217] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:20 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:20.948376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b03c5-0153-4953-9e9f-63ada6ea7f1f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:20.953568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:20.970968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b794e7a-249b-471f-a37f-148441d9d429 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:21.015779 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14018e01-e1ca-461b-9a30-a83f35c2423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:21.016463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14018e01-e1ca-461b-9a30-a83f35c2423a None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:21.016743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14018e01-e1ca-461b-9a30-a83f35c2423a None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:21.017013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14018e01-e1ca-461b-9a30-a83f35c2423a None admin] PATH_INFO: `/v3/users/503642c6603642ef9f94a00fb1aa4a2d` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:21.018204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14018e01-e1ca-461b-9a30-a83f35c2423a None admin] RBAC: Authorizing `identity:get_user(user_id=503642c6603642ef9f94a00fb1aa4a2d)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:21.019706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14018e01-e1ca-461b-9a30-a83f35c2423a None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:21.021528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 109/218] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:20 2022] GET /identity/v3/users/503642c6603642ef9f94a00fb1aa4a2d => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:21.027899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14018e01-e1ca-461b-9a30-a83f35c2423a None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:21.042774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:21.068404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b03c5-0153-4953-9e9f-63ada6ea7f1f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:21.108912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fb9d4-b581-4215-b7c3-2bb47c4e3b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:21.109552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fb9d4-b581-4215-b7c3-2bb47c4e3b80 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:21.109830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fb9d4-b581-4215-b7c3-2bb47c4e3b80 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:21.110102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fb9d4-b581-4215-b7c3-2bb47c4e3b80 None admin] PATH_INFO: `/v3/users/6b8d80f35625471fa4d58d296722b9bf` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:21.126458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fb9d4-b581-4215-b7c3-2bb47c4e3b80 None admin] RBAC: Authorizing `identity:get_user(user_id=6b8d80f35625471fa4d58d296722b9bf)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:21.127924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fb9d4-b581-4215-b7c3-2bb47c4e3b80 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:21.129688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 110/219] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:20 2022] GET /identity/v3/users/6b8d80f35625471fa4d58d296722b9bf => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:21.135426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fb9d4-b581-4215-b7c3-2bb47c4e3b80 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:21.147847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14018e01-e1ca-461b-9a30-a83f35c2423a None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:21.188784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095f6d-f19f-4444-b3e2-a67ef64d1660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:21.189393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095f6d-f19f-4444-b3e2-a67ef64d1660 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:21.189669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095f6d-f19f-4444-b3e2-a67ef64d1660 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:21.189940 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095f6d-f19f-4444-b3e2-a67ef64d1660 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:21.219245 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:21.256595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fb9d4-b581-4215-b7c3-2bb47c4e3b80 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:21.269449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5095f6d-f19f-4444-b3e2-a67ef64d1660 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:21.270937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5095f6d-f19f-4444-b3e2-a67ef64d1660 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:21.297988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf01ea3-0cb4-403e-9f07-af3589dc2d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:21.298971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf01ea3-0cb4-403e-9f07-af3589dc2d37 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:21.299479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf01ea3-0cb4-403e-9f07-af3589dc2d37 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:21.299953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf01ea3-0cb4-403e-9f07-af3589dc2d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:21.300955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf01ea3-0cb4-403e-9f07-af3589dc2d37 None admin] RBAC: Adding query filter params (role.id=a73215dcd05645c89a2f742200bbb0c9, user.id=6b8d80f35625471fa4d58d296722b9bf, scope.system=all) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:21.301478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf01ea3-0cb4-403e-9f07-af3589dc2d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:21.303195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf01ea3-0cb4-403e-9f07-af3589dc2d37 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:21.308530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:21.316370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 111/220] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Oct 20 14:28:21 2022] GET /identity/v3/role_assignments?user.id=6b8d80f35625471fa4d58d296722b9bf&scope.system=all&role.id=a73215dcd05645c89a2f742200bbb0c9 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:21.401859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 110/221] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:21.409768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf01ea3-0cb4-403e-9f07-af3589dc2d37 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:21.530273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf01ea3-0cb4-403e-9f07-af3589dc2d37 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:21.567545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e236a34-8332-4a0d-b33b-c3ab197538a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:21.568190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e236a34-8332-4a0d-b33b-c3ab197538a7 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:21.568504 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e236a34-8332-4a0d-b33b-c3ab197538a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:21.568783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e236a34-8332-4a0d-b33b-c3ab197538a7 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:21.569919 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e236a34-8332-4a0d-b33b-c3ab197538a7 None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:21.571390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e236a34-8332-4a0d-b33b-c3ab197538a7 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:21.573358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 112/222] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:21 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:21.579410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095f6d-f19f-4444-b3e2-a67ef64d1660 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:21.643579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:21.690916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095f6d-f19f-4444-b3e2-a67ef64d1660 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:21.726906 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000c6a1-a0eb-40a5-92e1-b9c8dd47ceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:21.727516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c6a1-a0eb-40a5-92e1-b9c8dd47ceb8 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:21.727792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c6a1-a0eb-40a5-92e1-b9c8dd47ceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:21.728061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c6a1-a0eb-40a5-92e1-b9c8dd47ceb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:21.728815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000c6a1-a0eb-40a5-92e1-b9c8dd47ceb8 None admin] RBAC: Adding query filter params (role.id=c8d552ca72b140a8a815448527770138, scope.project.id=6da9e929c8864aca8ed43b28bed3d346, user.id=503642c6603642ef9f94a00fb1aa4a2d) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:21.729119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000c6a1-a0eb-40a5-92e1-b9c8dd47ceb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:21.730635 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000c6a1-a0eb-40a5-92e1-b9c8dd47ceb8 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:21.737239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 111/223] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:21 2022] GET /identity/v3/role_assignments?user.id=503642c6603642ef9f94a00fb1aa4a2d&scope.project.id=6da9e929c8864aca8ed43b28bed3d346&role.id=c8d552ca72b140a8a815448527770138 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:23.289651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd4974-4205-43f6-8083-163ee27d82ad None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:23.290167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd4974-4205-43f6-8083-163ee27d82ad None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:23.290239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd4974-4205-43f6-8083-163ee27d82ad None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:23.291694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 113/224] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:23.297277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adecf70-8281-409b-bf45-eea88cb47d22 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:23.297682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adecf70-8281-409b-bf45-eea88cb47d22 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:23.297839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adecf70-8281-409b-bf45-eea88cb47d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:23.316637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-9adecf70-8281-409b-bf45-eea88cb47d22 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:23.317812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9adecf70-8281-409b-bf45-eea88cb47d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:23.333876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0261-34dd-42d8-bac7-654607723dbd None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:23.334172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0261-34dd-42d8-bac7-654607723dbd None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:23.334457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0261-34dd-42d8-bac7-654607723dbd None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:23.335952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 114/225] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:28:23.341380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c7c6-0d6a-42da-843a-bfe00566aa9a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:23.341663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c7c6-0d6a-42da-843a-bfe00566aa9a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:23.341941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c7c6-0d6a-42da-843a-bfe00566aa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:23.364473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4b90c7c6-0d6a-42da-843a-bfe00566aa9a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:23.365675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4b90c7c6-0d6a-42da-843a-bfe00566aa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:23.400859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 112/226] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:23.441587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:23.447951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 115/227] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:23.528823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255c437-66f5-4b9a-aa1e-36386710e849 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:23.529125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255c437-66f5-4b9a-aa1e-36386710e849 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:23.529375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255c437-66f5-4b9a-aa1e-36386710e849 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:23.530879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 113/228] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:23.535690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f67b4-b4d7-4cfa-937c-45723aaca17c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:23.535980 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f67b4-b4d7-4cfa-937c-45723aaca17c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:23.536255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f67b4-b4d7-4cfa-937c-45723aaca17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:23.554643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-658f67b4-b4d7-4cfa-937c-45723aaca17c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:23.555739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-658f67b4-b4d7-4cfa-937c-45723aaca17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:23.621032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 116/229] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:23.674424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255c437-66f5-4b9a-aa1e-36386710e849 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:23.722150 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f67b4-b4d7-4cfa-937c-45723aaca17c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:23.729079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:23.782948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255c437-66f5-4b9a-aa1e-36386710e849 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:23.817518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed705a4-cd73-4a49-8d93-59fcf6abcaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:23.818168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed705a4-cd73-4a49-8d93-59fcf6abcaa0 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:23.818443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed705a4-cd73-4a49-8d93-59fcf6abcaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:23.818715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed705a4-cd73-4a49-8d93-59fcf6abcaa0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:23.823561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f67b4-b4d7-4cfa-937c-45723aaca17c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:23.830012 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed705a4-cd73-4a49-8d93-59fcf6abcaa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-1ed705a4-cd73-4a49-8d93-59fcf6abcaa0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:23.835275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:23.836116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:23.836943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 114/230] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:23.842186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed705a4-cd73-4a49-8d93-59fcf6abcaa0 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:23.854801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:23.863649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988aef-5012-42d4-bc80-f21c8e19e5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:23.864282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988aef-5012-42d4-bc80-f21c8e19e5a5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:23.864588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988aef-5012-42d4-bc80-f21c8e19e5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:23.864858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988aef-5012-42d4-bc80-f21c8e19e5a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:23.865954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78988aef-5012-42d4-bc80-f21c8e19e5a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:23.867387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78988aef-5012-42d4-bc80-f21c8e19e5a5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:23.869387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 117/231] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:23.876060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988aef-5012-42d4-bc80-f21c8e19e5a5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:23.947659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed705a4-cd73-4a49-8d93-59fcf6abcaa0 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:23.958428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:23.979770 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988aef-5012-42d4-bc80-f21c8e19e5a5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:23.982823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4b0f7-4913-4780-a03f-640f0c10fda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:23.983423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4b0f7-4913-4780-a03f-640f0c10fda0 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:23.983703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4b0f7-4913-4780-a03f-640f0c10fda0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:23.983977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4b0f7-4913-4780-a03f-640f0c10fda0 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:23.984666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4b0f7-4913-4780-a03f-640f0c10fda0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:23.984971 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4b0f7-4913-4780-a03f-640f0c10fda0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:23.986371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4b0f7-4913-4780-a03f-640f0c10fda0 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:23.994582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 115/232] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:23.995448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4b0f7-4913-4780-a03f-640f0c10fda0 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:24.018674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31117b7d-617b-42a8-9011-cdd37f0ad53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:24.019283 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31117b7d-617b-42a8-9011-cdd37f0ad53b None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:24.019564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31117b7d-617b-42a8-9011-cdd37f0ad53b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:24.019840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31117b7d-617b-42a8-9011-cdd37f0ad53b None admin] PATH_INFO: `/v3/groups` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:24.020511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31117b7d-617b-42a8-9011-cdd37f0ad53b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:24.021943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31117b7d-617b-42a8-9011-cdd37f0ad53b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:24.047043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 118/233] 10.222.0.13 () {60 vars in 1134 bytes} [Thu Oct 20 14:28:23 2022] POST /identity/v3/groups => generated 229 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:28:24.047043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31117b7d-617b-42a8-9011-cdd37f0ad53b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:24.103474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:24.117803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4b0f7-4913-4780-a03f-640f0c10fda0 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:24.155523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760a7b6-fa14-4650-a833-aae00a7afae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:24.156179 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760a7b6-fa14-4650-a833-aae00a7afae2 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:24.156495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760a7b6-fa14-4650-a833-aae00a7afae2 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:24.156774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760a7b6-fa14-4650-a833-aae00a7afae2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:24.159106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31117b7d-617b-42a8-9011-cdd37f0ad53b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:24.169154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760a7b6-fa14-4650-a833-aae00a7afae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-b760a7b6-fa14-4650-a833-aae00a7afae2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:24.174776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:24.175706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:24.176507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 116/234] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:24.182163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760a7b6-fa14-4650-a833-aae00a7afae2 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:24.201868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929c5f6-d3b6-4fe4-a11b-cc6bb5cf3dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:24.202507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929c5f6-d3b6-4fe4-a11b-cc6bb5cf3dbc None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:24.202785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929c5f6-d3b6-4fe4-a11b-cc6bb5cf3dbc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:24.203090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929c5f6-d3b6-4fe4-a11b-cc6bb5cf3dbc None admin] PATH_INFO: `/v3/users/20aa921d7fd344e399a8c3bb0e929355` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:24.218363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929c5f6-d3b6-4fe4-a11b-cc6bb5cf3dbc None admin] RBAC: Authorizing `identity:get_user(user_id=20aa921d7fd344e399a8c3bb0e929355)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:24.218572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:24.219866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929c5f6-d3b6-4fe4-a11b-cc6bb5cf3dbc None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:24.221752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 119/235] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:24 2022] GET /identity/v3/users/20aa921d7fd344e399a8c3bb0e929355 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:24.227401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929c5f6-d3b6-4fe4-a11b-cc6bb5cf3dbc None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:24.301132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760a7b6-fa14-4650-a833-aae00a7afae2 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:24.335244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929c5f6-d3b6-4fe4-a11b-cc6bb5cf3dbc None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:24.339539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57be457-9fc2-47b1-953f-37208c557763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:24.340606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57be457-9fc2-47b1-953f-37208c557763 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:24.341042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57be457-9fc2-47b1-953f-37208c557763 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:24.341426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57be457-9fc2-47b1-953f-37208c557763 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:24.342215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57be457-9fc2-47b1-953f-37208c557763 None admin] RBAC: Adding query filter params (name=member) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:24.342627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57be457-9fc2-47b1-953f-37208c557763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:24.344042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57be457-9fc2-47b1-953f-37208c557763 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:24.353805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:24.361187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 117/236] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:24.362597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57be457-9fc2-47b1-953f-37208c557763 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:24.377416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af9d03-82d5-4dc7-801d-0142925898e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:24.378276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af9d03-82d5-4dc7-801d-0142925898e8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:24.378556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af9d03-82d5-4dc7-801d-0142925898e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:24.378999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af9d03-82d5-4dc7-801d-0142925898e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:24.380091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af9d03-82d5-4dc7-801d-0142925898e8 None admin] RBAC: Adding query filter params (role.id=c8d552ca72b140a8a815448527770138, user.id=20aa921d7fd344e399a8c3bb0e929355, scope.system=all) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:24.380443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af9d03-82d5-4dc7-801d-0142925898e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:24.382060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af9d03-82d5-4dc7-801d-0142925898e8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:24.387834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 120/237] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Oct 20 14:28:24 2022] GET /identity/v3/role_assignments?user.id=20aa921d7fd344e399a8c3bb0e929355&scope.system=all&role.id=c8d552ca72b140a8a815448527770138 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:24.470597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:24.479032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57be457-9fc2-47b1-953f-37208c557763 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:24.517145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f372b47-e91c-42ad-8c83-f7e5526d5817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:24.517823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f372b47-e91c-42ad-8c83-f7e5526d5817 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:24.518108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f372b47-e91c-42ad-8c83-f7e5526d5817 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:24.518381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f372b47-e91c-42ad-8c83-f7e5526d5817 None admin] PATH_INFO: `/v3/users/6b8d80f35625471fa4d58d296722b9bf` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:24.519534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f372b47-e91c-42ad-8c83-f7e5526d5817 None admin] RBAC: Authorizing `identity:get_user(user_id=6b8d80f35625471fa4d58d296722b9bf)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:24.521055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f372b47-e91c-42ad-8c83-f7e5526d5817 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:24.522837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 118/238] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:24 2022] GET /identity/v3/users/6b8d80f35625471fa4d58d296722b9bf => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:24.529682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af9d03-82d5-4dc7-801d-0142925898e8 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:24.636745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af9d03-82d5-4dc7-801d-0142925898e8 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:24.670343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abfa6f-a9d6-4efe-97fd-75b48eab1f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:24.670993 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abfa6f-a9d6-4efe-97fd-75b48eab1f0d None admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:24.671273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abfa6f-a9d6-4efe-97fd-75b48eab1f0d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:24.671548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abfa6f-a9d6-4efe-97fd-75b48eab1f0d None admin] PATH_INFO: `/v3/system/users/6b8d80f35625471fa4d58d296722b9bf/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:24.672967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abfa6f-a9d6-4efe-97fd-75b48eab1f0d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6b8d80f35625471fa4d58d296722b9bf, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:24.674352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abfa6f-a9d6-4efe-97fd-75b48eab1f0d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:24.683044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 121/239] 10.222.0.13 () {58 vars in 1334 bytes} [Thu Oct 20 14:28:24 2022] PUT /identity/v3/system/users/6b8d80f35625471fa4d58d296722b9bf/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:26.213406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4565-8dc2-4a3b-ab37-00372b14bb80 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:26.213923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4565-8dc2-4a3b-ab37-00372b14bb80 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:26.213996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4565-8dc2-4a3b-ab37-00372b14bb80 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:26.216591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 119/240] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:26.222015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d2b47-9b80-47f6-936e-3e0643310a74 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:26.222420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d2b47-9b80-47f6-936e-3e0643310a74 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:26.222594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d2b47-9b80-47f6-936e-3e0643310a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:26.240954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-878d2b47-9b80-47f6-936e-3e0643310a74 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:26.242113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-878d2b47-9b80-47f6-936e-3e0643310a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:26.276887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:26.311375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 122/241] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:26.536494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439d1ca-f338-428b-b97a-5a23f17a92c4 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:26.536816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439d1ca-f338-428b-b97a-5a23f17a92c4 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:26.537006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439d1ca-f338-428b-b97a-5a23f17a92c4 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:26.538340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 120/242] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:26.544079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4010c-1377-425d-bbe5-d8787a023872 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:26.544374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4010c-1377-425d-bbe5-d8787a023872 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:26.544654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4010c-1377-425d-bbe5-d8787a023872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:26.562907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-dfb4010c-1377-425d-bbe5-d8787a023872 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:26.564071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-dfb4010c-1377-425d-bbe5-d8787a023872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:26.579045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439d1ca-f338-428b-b97a-5a23f17a92c4 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:26.632459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 123/243] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:26.688877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439d1ca-f338-428b-b97a-5a23f17a92c4 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:26.725235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228171f-46a6-4da5-9205-8d8b732670e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:26.725886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228171f-46a6-4da5-9205-8d8b732670e7 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:26.726239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228171f-46a6-4da5-9205-8d8b732670e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:26.726515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228171f-46a6-4da5-9205-8d8b732670e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:26.727683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228171f-46a6-4da5-9205-8d8b732670e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:26.729297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228171f-46a6-4da5-9205-8d8b732670e7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:26.731525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 121/244] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:26.737118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4010c-1377-425d-bbe5-d8787a023872 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:26.827892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7887b1-563a-472c-b236-6013c3ce3b97 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:26.828178 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7887b1-563a-472c-b236-6013c3ce3b97 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:26.828397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7887b1-563a-472c-b236-6013c3ce3b97 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:26.829884 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 122/245] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:28:26.834540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c90d8-8126-4d7e-813d-3e4d26c11b5e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:26.834830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c90d8-8126-4d7e-813d-3e4d26c11b5e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:26.835109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c90d8-8126-4d7e-813d-3e4d26c11b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:26.837224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4010c-1377-425d-bbe5-d8787a023872 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:26.838808 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:26.854483 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-9a5c90d8-8126-4d7e-813d-3e4d26c11b5e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:26.856461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9a5c90d8-8126-4d7e-813d-3e4d26c11b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:26.874335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373510f-6a69-44ef-a622-df71bae7581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:26.875082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373510f-6a69-44ef-a622-df71bae7581a None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:26.875652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373510f-6a69-44ef-a622-df71bae7581a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:26.875930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373510f-6a69-44ef-a622-df71bae7581a None admin] PATH_INFO: `/v3/groups` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:26.876620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373510f-6a69-44ef-a622-df71bae7581a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:26.878510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373510f-6a69-44ef-a622-df71bae7581a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:26.881615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:26.900377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 124/246] 10.222.0.13 () {60 vars in 1134 bytes} [Thu Oct 20 14:28:26 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:28:26.901916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373510f-6a69-44ef-a622-df71bae7581a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:26.934800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 123/247] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:27.017885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373510f-6a69-44ef-a622-df71bae7581a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:27.055298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f253bd-f791-4f67-acc4-7ee35c309730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:27.056370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f253bd-f791-4f67-acc4-7ee35c309730 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:27.056882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f253bd-f791-4f67-acc4-7ee35c309730 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:27.057365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f253bd-f791-4f67-acc4-7ee35c309730 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:27.070255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f253bd-f791-4f67-acc4-7ee35c309730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-29f253bd-f791-4f67-acc4-7ee35c309730 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:27.076407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:27.077258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:27.078230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 125/248] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:27.083377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c90d8-8126-4d7e-813d-3e4d26c11b5e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:27.192104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c90d8-8126-4d7e-813d-3e4d26c11b5e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:27.203787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f253bd-f791-4f67-acc4-7ee35c309730 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:27.232737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96a6a3-6d72-4e96-a42e-8a8bfc2e76f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:27.233275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:27.234727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96a6a3-6d72-4e96-a42e-8a8bfc2e76f0 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:27.235181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96a6a3-6d72-4e96-a42e-8a8bfc2e76f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:27.235511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96a6a3-6d72-4e96-a42e-8a8bfc2e76f0 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:27.236196 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96a6a3-6d72-4e96-a42e-8a8bfc2e76f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:27.236530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96a6a3-6d72-4e96-a42e-8a8bfc2e76f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:27.237931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96a6a3-6d72-4e96-a42e-8a8bfc2e76f0 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:27.250441 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 124/249] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:27.255449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96a6a3-6d72-4e96-a42e-8a8bfc2e76f0 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:27.288987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:27.330157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f253bd-f791-4f67-acc4-7ee35c309730 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:27.389671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0f2b9-dc41-4160-a74c-b32a8a717647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:27.390661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0f2b9-dc41-4160-a74c-b32a8a717647 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:27.391138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0f2b9-dc41-4160-a74c-b32a8a717647 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:27.391607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0f2b9-dc41-4160-a74c-b32a8a717647 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:27.410249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96a6a3-6d72-4e96-a42e-8a8bfc2e76f0 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:27.416275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0f2b9-dc41-4160-a74c-b32a8a717647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-d5e0f2b9-dc41-4160-a74c-b32a8a717647 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:27.423106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:27.424019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:27.424819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 126/250] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:27.431897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0f2b9-dc41-4160-a74c-b32a8a717647 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:27.462989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0b568-2dcf-4b0d-b073-1aa4d6e3401c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:27.463998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0b568-2dcf-4b0d-b073-1aa4d6e3401c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:27.464678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0b568-2dcf-4b0d-b073-1aa4d6e3401c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:27.465690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0b568-2dcf-4b0d-b073-1aa4d6e3401c None admin] PATH_INFO: `/v3/users/20aa921d7fd344e399a8c3bb0e929355` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:27.467109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0b568-2dcf-4b0d-b073-1aa4d6e3401c None admin] RBAC: Authorizing `identity:get_user(user_id=20aa921d7fd344e399a8c3bb0e929355)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:27.468836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0b568-2dcf-4b0d-b073-1aa4d6e3401c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:27.470972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 125/251] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:27 2022] GET /identity/v3/users/20aa921d7fd344e399a8c3bb0e929355 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:27.477331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0b568-2dcf-4b0d-b073-1aa4d6e3401c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:27.556406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0f2b9-dc41-4160-a74c-b32a8a717647 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:27.560700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:27.582860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:27.595023 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0b568-2dcf-4b0d-b073-1aa4d6e3401c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:27.602894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecaba6-22c6-491b-9964-812d57fd41c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:27.603949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecaba6-22c6-491b-9964-812d57fd41c7 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:27.604237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecaba6-22c6-491b-9964-812d57fd41c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:27.604547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecaba6-22c6-491b-9964-812d57fd41c7 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:27.605706 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecaba6-22c6-491b-9964-812d57fd41c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:27.606027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecaba6-22c6-491b-9964-812d57fd41c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:27.607791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecaba6-22c6-491b-9964-812d57fd41c7 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:27.618513 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 127/252] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:27.624947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecaba6-22c6-491b-9964-812d57fd41c7 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:27.640665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b8727-7afd-48cd-b9e0-b900c27b22ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:27.641438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b8727-7afd-48cd-b9e0-b900c27b22ea None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:27.641815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b8727-7afd-48cd-b9e0-b900c27b22ea None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:27.642095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b8727-7afd-48cd-b9e0-b900c27b22ea None admin] PATH_INFO: `/v3/system/users/20aa921d7fd344e399a8c3bb0e929355/roles/c8d552ca72b140a8a815448527770138` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:27.644269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b8727-7afd-48cd-b9e0-b900c27b22ea None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=20aa921d7fd344e399a8c3bb0e929355, role_id=c8d552ca72b140a8a815448527770138)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:27.645901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b8727-7afd-48cd-b9e0-b900c27b22ea None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:27.661005 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 126/253] 10.222.0.13 () {58 vars in 1334 bytes} [Thu Oct 20 14:28:27 2022] PUT /identity/v3/system/users/20aa921d7fd344e399a8c3bb0e929355/roles/c8d552ca72b140a8a815448527770138 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:28:27.747654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecaba6-22c6-491b-9964-812d57fd41c7 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:27.787257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85806a65-3a20-4b60-a8c2-918163e4dae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:27.787919 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806a65-3a20-4b60-a8c2-918163e4dae8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:27.788207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806a65-3a20-4b60-a8c2-918163e4dae8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:27.788530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806a65-3a20-4b60-a8c2-918163e4dae8 None admin] PATH_INFO: `/v3/users/6b8d80f35625471fa4d58d296722b9bf` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:27.789758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85806a65-3a20-4b60-a8c2-918163e4dae8 None admin] RBAC: Authorizing `identity:get_user(user_id=6b8d80f35625471fa4d58d296722b9bf)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:27.791227 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85806a65-3a20-4b60-a8c2-918163e4dae8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:27.793034 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 128/254] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:27 2022] GET /identity/v3/users/6b8d80f35625471fa4d58d296722b9bf => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:27.799129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b8727-7afd-48cd-b9e0-b900c27b22ea None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:27.932007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b8727-7afd-48cd-b9e0-b900c27b22ea None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:27.979879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfc2f5-b64a-450e-b0bb-e4308026e9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:27.981174 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfc2f5-b64a-450e-b0bb-e4308026e9bd None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:27.981458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfc2f5-b64a-450e-b0bb-e4308026e9bd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:27.981734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfc2f5-b64a-450e-b0bb-e4308026e9bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:27.982528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfc2f5-b64a-450e-b0bb-e4308026e9bd None admin] RBAC: Adding query filter params (role.id=a73215dcd05645c89a2f742200bbb0c9, user.id=6b8d80f35625471fa4d58d296722b9bf, scope.system=all) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:27.982835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfc2f5-b64a-450e-b0bb-e4308026e9bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:27.984566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfc2f5-b64a-450e-b0bb-e4308026e9bd None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:28.001934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 127/255] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Oct 20 14:28:27 2022] GET /identity/v3/role_assignments?user.id=6b8d80f35625471fa4d58d296722b9bf&scope.system=all&role.id=a73215dcd05645c89a2f742200bbb0c9 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:29.466293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01c8f-6a01-463e-abd1-a35db86c4449 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:29.466823 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01c8f-6a01-463e-abd1-a35db86c4449 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:29.466823 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01c8f-6a01-463e-abd1-a35db86c4449 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:29.468456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 129/256] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:29.476850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4c502-92d2-4eab-8a1d-cfb6acf9bffe None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:29.477297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4c502-92d2-4eab-8a1d-cfb6acf9bffe None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:29.477391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4c502-92d2-4eab-8a1d-cfb6acf9bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:29.499417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-0fe4c502-92d2-4eab-8a1d-cfb6acf9bffe None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:29.500671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0fe4c502-92d2-4eab-8a1d-cfb6acf9bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:29.511333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:29.576958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 128/257] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 20 14:28:29.969250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af70b6-64ee-4f21-8a03-d606d7202bfe None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:29.969662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af70b6-64ee-4f21-8a03-d606d7202bfe None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:29.969831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af70b6-64ee-4f21-8a03-d606d7202bfe None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:29.971404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 130/258] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:29.976749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b0f8e-8d0d-4857-ac91-a980831ecdec None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:29.977042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b0f8e-8d0d-4857-ac91-a980831ecdec None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:29.977327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b0f8e-8d0d-4857-ac91-a980831ecdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.002565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-c44b0f8e-8d0d-4857-ac91-a980831ecdec None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:30.003782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c44b0f8e-8d0d-4857-ac91-a980831ecdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:30.014645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 129/259] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 14:28:30.135116 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af70b6-64ee-4f21-8a03-d606d7202bfe None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.155396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af70b6-64ee-4f21-8a03-d606d7202bfe None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.159917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e12a24-319a-4a6e-a617-c5b4223bbe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.160581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e12a24-319a-4a6e-a617-c5b4223bbe00 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.160866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e12a24-319a-4a6e-a617-c5b4223bbe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.161214 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e12a24-319a-4a6e-a617-c5b4223bbe00 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.173201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:30.177220 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e12a24-319a-4a6e-a617-c5b4223bbe00 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-a4e12a24-319a-4a6e-a617-c5b4223bbe00 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:30.182203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:30.184143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:30.185857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:30.188654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 131/260] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Oct 20 14:28:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:30.189339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b0f8e-8d0d-4857-ac91-a980831ecdec None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.203878 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b0f8e-8d0d-4857-ac91-a980831ecdec None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.211935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.212577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.212860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.213138 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.213811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] RBAC: Adding query filter params (name=member) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:30.214116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:30.215974 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING py.warnings [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:28:30.215974 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: warnings.warn(msg) Oct 20 14:28:30.215974 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: Oct 20 14:28:30.216909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:30.233446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 130/261] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:28:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:30.238610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e12a24-319a-4a6e-a617-c5b4223bbe00 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.251816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e12a24-319a-4a6e-a617-c5b4223bbe00 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.260449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59aa780-03f3-41ed-9028-0fbe2db805ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.261085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa780-03f3-41ed-9028-0fbe2db805ef admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.261880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa780-03f3-41ed-9028-0fbe2db805ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.262230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa780-03f3-41ed-9028-0fbe2db805ef admin admin] PATH_INFO: `/v3/groups/cef2035dbfdd43e496229c29189164c5` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.274887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59aa780-03f3-41ed-9028-0fbe2db805ef admin admin] RBAC: Authorizing `identity:get_group(group_id=cef2035dbfdd43e496229c29189164c5)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:30.276734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING py.warnings [None req-d59aa780-03f3-41ed-9028-0fbe2db805ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:28:30.276734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: warnings.warn(msg) Oct 20 14:28:30.276734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: Oct 20 14:28:30.278179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59aa780-03f3-41ed-9028-0fbe2db805ef admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:30.280155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 132/262] 10.222.0.13 () {56 vars in 1201 bytes} [Thu Oct 20 14:28:30 2022] GET /identity/v3/groups/cef2035dbfdd43e496229c29189164c5 => generated 226 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:30.285303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.302999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5a03e-998a-44b0-86f1-e853ca004d6e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.307497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a06b2a-29b0-435a-ba06-ff37ba4c5bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.308142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a06b2a-29b0-435a-ba06-ff37ba4c5bcf admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.308534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a06b2a-29b0-435a-ba06-ff37ba4c5bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.308817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a06b2a-29b0-435a-ba06-ff37ba4c5bcf admin admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.309958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a06b2a-29b0-435a-ba06-ff37ba4c5bcf admin admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:30.312035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a06b2a-29b0-435a-ba06-ff37ba4c5bcf admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:30.313703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 131/263] 10.222.0.13 () {56 vars in 1207 bytes} [Thu Oct 20 14:28:30 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:30.318258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736863f6-62c7-482e-9868-27b4a5f20058 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.318547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736863f6-62c7-482e-9868-27b4a5f20058 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.318829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736863f6-62c7-482e-9868-27b4a5f20058 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.319113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a06b2a-29b0-435a-ba06-ff37ba4c5bcf admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.320148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 133/264] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:30.330053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc82e5-d594-47a3-9d35-eef6bdb203c9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.330320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc82e5-d594-47a3-9d35-eef6bdb203c9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.330606 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc82e5-d594-47a3-9d35-eef6bdb203c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.333820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a06b2a-29b0-435a-ba06-ff37ba4c5bcf admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.341728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536363f-90cc-4efc-a836-0a4c886f819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.341728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536363f-90cc-4efc-a836-0a4c886f819b admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.341728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536363f-90cc-4efc-a836-0a4c886f819b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.341728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536363f-90cc-4efc-a836-0a4c886f819b admin admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/groups/cef2035dbfdd43e496229c29189164c5/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.342414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536363f-90cc-4efc-a836-0a4c886f819b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8dde0c3629144eabb2ec9e6e6c16a81, group_id=cef2035dbfdd43e496229c29189164c5, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:30.349186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING py.warnings [None req-3536363f-90cc-4efc-a836-0a4c886f819b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:28:30.349186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: warnings.warn(msg) Oct 20 14:28:30.349186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: Oct 20 14:28:30.349186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536363f-90cc-4efc-a836-0a4c886f819b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:30.364040 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 132/265] 10.222.0.13 () {58 vars in 1463 bytes} [Thu Oct 20 14:28:30 2022] PUT /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/groups/cef2035dbfdd43e496229c29189164c5/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:28:30.377531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f2dc82e5-d594-47a3-9d35-eef6bdb203c9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:30.378776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f2dc82e5-d594-47a3-9d35-eef6bdb203c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:30.470798 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 134/266] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 20 14:28:30.498144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536363f-90cc-4efc-a836-0a4c886f819b admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.577625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536363f-90cc-4efc-a836-0a4c886f819b admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.582554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d6e40-7332-429a-ba9b-a17728487ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.583477 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d6e40-7332-429a-ba9b-a17728487ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.583949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d6e40-7332-429a-ba9b-a17728487ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.584470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d6e40-7332-429a-ba9b-a17728487ba4 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.597599 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d6e40-7332-429a-ba9b-a17728487ba4 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-887d6e40-7332-429a-ba9b-a17728487ba4 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:30.603021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:30.603980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:30.604860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:30.607748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 133/267] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Oct 20 14:28:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 110 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:30.613715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc82e5-d594-47a3-9d35-eef6bdb203c9 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.637591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc82e5-d594-47a3-9d35-eef6bdb203c9 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.646119 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.646755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.647036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.647312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.648014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:30.648322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:30.649422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING py.warnings [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:28:30.649422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: warnings.warn(msg) Oct 20 14:28:30.649422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: Oct 20 14:28:30.650358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:30.668041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 135/268] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Oct 20 14:28:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:30.674299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d6e40-7332-429a-ba9b-a17728487ba4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.690180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d6e40-7332-429a-ba9b-a17728487ba4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.698814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32741d6b-328f-472c-b324-563a98959ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.700228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32741d6b-328f-472c-b324-563a98959ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.700735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32741d6b-328f-472c-b324-563a98959ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.701306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32741d6b-328f-472c-b324-563a98959ef4 admin admin] PATH_INFO: `/v3/groups/cef2035dbfdd43e496229c29189164c5` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.705059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32741d6b-328f-472c-b324-563a98959ef4 admin admin] RBAC: Authorizing `identity:get_group(group_id=cef2035dbfdd43e496229c29189164c5)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:30.706343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING py.warnings [None req-32741d6b-328f-472c-b324-563a98959ef4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:28:30.706343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: warnings.warn(msg) Oct 20 14:28:30.706343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: Oct 20 14:28:30.708090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32741d6b-328f-472c-b324-563a98959ef4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:30.710080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 134/269] 10.222.0.13 () {56 vars in 1201 bytes} [Thu Oct 20 14:28:30 2022] GET /identity/v3/groups/cef2035dbfdd43e496229c29189164c5 => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:30.716287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.730827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241d717-cac2-44a1-aba7-faf73f2178c3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.738888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7248d-701b-4d6c-807e-7a6099b5a0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.739543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7248d-701b-4d6c-807e-7a6099b5a0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.739827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7248d-701b-4d6c-807e-7a6099b5a0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.740118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7248d-701b-4d6c-807e-7a6099b5a0f1 admin admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.741288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7248d-701b-4d6c-807e-7a6099b5a0f1 admin admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:30.743650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7248d-701b-4d6c-807e-7a6099b5a0f1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:30.745628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 136/270] 10.222.0.13 () {56 vars in 1207 bytes} [Thu Oct 20 14:28:30 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:30.751783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32741d6b-328f-472c-b324-563a98959ef4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:30.766248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32741d6b-328f-472c-b324-563a98959ef4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:30.774235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af785af6-6d77-46a5-ad79-68dcf980b817 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.774563 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af785af6-6d77-46a5-ad79-68dcf980b817 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.774793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af785af6-6d77-46a5-ad79-68dcf980b817 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.775992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666b6c0-e160-4a20-9e16-78d3f5576cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:30.776421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 137/271] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:28:30.777699 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666b6c0-e160-4a20-9e16-78d3f5576cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.778236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666b6c0-e160-4a20-9e16-78d3f5576cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.778696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666b6c0-e160-4a20-9e16-78d3f5576cd5 admin admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/groups/cef2035dbfdd43e496229c29189164c5/roles/dff349185bc94cb4a0e0f0c40425862f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.782842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288d5ed-3378-43d5-b312-1069a943c776 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:30.783136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288d5ed-3378-43d5-b312-1069a943c776 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:30.783419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288d5ed-3378-43d5-b312-1069a943c776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:30.784089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666b6c0-e160-4a20-9e16-78d3f5576cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8dde0c3629144eabb2ec9e6e6c16a81, group_id=cef2035dbfdd43e496229c29189164c5, role_id=dff349185bc94cb4a0e0f0c40425862f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:30.787175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666b6c0-e160-4a20-9e16-78d3f5576cd5 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:30.809957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-7288d5ed-3378-43d5-b312-1069a943c776 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:30.811544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7288d5ed-3378-43d5-b312-1069a943c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:30.813890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 135/272] 10.222.0.13 () {58 vars in 1463 bytes} [Thu Oct 20 14:28:30 2022] PUT /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81/groups/cef2035dbfdd43e496229c29189164c5/roles/dff349185bc94cb4a0e0f0c40425862f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:30.893664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 138/273] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 14:28:31.074581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666b6c0-e160-4a20-9e16-78d3f5576cd5 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:31.137185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:31.201286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666b6c0-e160-4a20-9e16-78d3f5576cd5 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:31.254628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca39d37-c293-4a4e-b858-25986e4573e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:31.255283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca39d37-c293-4a4e-b858-25986e4573e9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.255566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca39d37-c293-4a4e-b858-25986e4573e9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.255842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca39d37-c293-4a4e-b858-25986e4573e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.269256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca39d37-c293-4a4e-b858-25986e4573e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-7ca39d37-c293-4a4e-b858-25986e4573e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:31.274871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:31.275752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:31.276562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 136/274] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:31.282429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288d5ed-3378-43d5-b312-1069a943c776 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:31.347062 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:31.404902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288d5ed-3378-43d5-b312-1069a943c776 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:31.451965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641bcb-9a95-44e6-a664-7b1f322d1bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:31.452712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641bcb-9a95-44e6-a664-7b1f322d1bf5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.453066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641bcb-9a95-44e6-a664-7b1f322d1bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.453402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641bcb-9a95-44e6-a664-7b1f322d1bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.454189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f641bcb-9a95-44e6-a664-7b1f322d1bf5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:31.454491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f641bcb-9a95-44e6-a664-7b1f322d1bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:31.455971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f641bcb-9a95-44e6-a664-7b1f322d1bf5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:31.468439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 139/275] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:31.474198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca39d37-c293-4a4e-b858-25986e4573e9 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:31.532226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641bcb-9a95-44e6-a664-7b1f322d1bf5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:31.548561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641bcb-9a95-44e6-a664-7b1f322d1bf5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:31.561118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a80c31-14e9-4adf-b02b-f416ec93a020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:31.561758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a80c31-14e9-4adf-b02b-f416ec93a020 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.562033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a80c31-14e9-4adf-b02b-f416ec93a020 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.562300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a80c31-14e9-4adf-b02b-f416ec93a020 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.581376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a80c31-14e9-4adf-b02b-f416ec93a020 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-e6a80c31-14e9-4adf-b02b-f416ec93a020 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:31.586668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:31.587534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:31.588295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:31.592818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 140/276] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Oct 20 14:28:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 61 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:31.600816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a80c31-14e9-4adf-b02b-f416ec93a020 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:31.615200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca39d37-c293-4a4e-b858-25986e4573e9 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:31.618946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a80c31-14e9-4adf-b02b-f416ec93a020 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:31.632683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:31.645052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4921a2-7fbc-4dfd-97ed-0eb7da45c74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:31.645688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4921a2-7fbc-4dfd-97ed-0eb7da45c74b admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.645950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4921a2-7fbc-4dfd-97ed-0eb7da45c74b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.646223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4921a2-7fbc-4dfd-97ed-0eb7da45c74b admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.646891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4921a2-7fbc-4dfd-97ed-0eb7da45c74b admin admin] RBAC: Adding query filter params (name=member) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:31.647189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4921a2-7fbc-4dfd-97ed-0eb7da45c74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:31.648837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4921a2-7fbc-4dfd-97ed-0eb7da45c74b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:31.666451 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 141/277] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:28:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:31.681436 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4921a2-7fbc-4dfd-97ed-0eb7da45c74b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:31.681950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f57b5d-df89-4b89-a984-a8449c240cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:31.682611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f57b5d-df89-4b89-a984-a8449c240cff None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.682897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f57b5d-df89-4b89-a984-a8449c240cff None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.683269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f57b5d-df89-4b89-a984-a8449c240cff None admin] PATH_INFO: `/v3/users/20aa921d7fd344e399a8c3bb0e929355` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.684660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f57b5d-df89-4b89-a984-a8449c240cff None admin] RBAC: Authorizing `identity:get_user(user_id=20aa921d7fd344e399a8c3bb0e929355)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:31.686203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f57b5d-df89-4b89-a984-a8449c240cff None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:31.688203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 137/278] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:28:31 2022] GET /identity/v3/users/20aa921d7fd344e399a8c3bb0e929355 => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:31.696045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4921a2-7fbc-4dfd-97ed-0eb7da45c74b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:31.701728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f57b5d-df89-4b89-a984-a8449c240cff None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:31.705016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba19681-5406-4237-b4a9-8e26278476b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:31.705661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba19681-5406-4237-b4a9-8e26278476b5 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.705939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba19681-5406-4237-b4a9-8e26278476b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.706206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba19681-5406-4237-b4a9-8e26278476b5 admin admin] PATH_INFO: `/v3/groups/cef2035dbfdd43e496229c29189164c5` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.713246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba19681-5406-4237-b4a9-8e26278476b5 admin admin] RBAC: Authorizing `identity:get_group(group_id=cef2035dbfdd43e496229c29189164c5)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:31.715602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba19681-5406-4237-b4a9-8e26278476b5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:31.717634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 142/279] 10.222.0.13 () {56 vars in 1201 bytes} [Thu Oct 20 14:28:31 2022] GET /identity/v3/groups/cef2035dbfdd43e496229c29189164c5 => generated 226 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:31.728336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba19681-5406-4237-b4a9-8e26278476b5 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:31.744252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba19681-5406-4237-b4a9-8e26278476b5 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:31.762169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332278b-1bf7-4f9a-ab64-ca9106d699b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:31.762814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332278b-1bf7-4f9a-ab64-ca9106d699b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.763091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332278b-1bf7-4f9a-ab64-ca9106d699b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.763360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332278b-1bf7-4f9a-ab64-ca9106d699b3 admin admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.769130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332278b-1bf7-4f9a-ab64-ca9106d699b3 admin admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:31.771199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332278b-1bf7-4f9a-ab64-ca9106d699b3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:31.773089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 143/280] 10.222.0.13 () {56 vars in 1207 bytes} [Thu Oct 20 14:28:31 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:31.784096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332278b-1bf7-4f9a-ab64-ca9106d699b3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:31.803888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332278b-1bf7-4f9a-ab64-ca9106d699b3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:31.821032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f9bf5-801b-4a94-b7af-3ccdb0b3531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:31.821705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f9bf5-801b-4a94-b7af-3ccdb0b3531f admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.821985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f9bf5-801b-4a94-b7af-3ccdb0b3531f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.822262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f9bf5-801b-4a94-b7af-3ccdb0b3531f admin admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346/groups/cef2035dbfdd43e496229c29189164c5/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.829148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f9bf5-801b-4a94-b7af-3ccdb0b3531f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da9e929c8864aca8ed43b28bed3d346, group_id=cef2035dbfdd43e496229c29189164c5, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:31.830189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING py.warnings [None req-cc4f9bf5-801b-4a94-b7af-3ccdb0b3531f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:28:31.830189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: warnings.warn(msg) Oct 20 14:28:31.830189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: Oct 20 14:28:31.832101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f9bf5-801b-4a94-b7af-3ccdb0b3531f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:31.871854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 144/281] 10.222.0.13 () {58 vars in 1463 bytes} [Thu Oct 20 14:28:31 2022] PUT /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346/groups/cef2035dbfdd43e496229c29189164c5/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:28:31.877122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f57b5d-df89-4b89-a984-a8449c240cff None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:31.893004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24a94b-e5de-430e-812a-e06dbc13957b None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.893322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24a94b-e5de-430e-812a-e06dbc13957b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.893625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24a94b-e5de-430e-812a-e06dbc13957b None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.898640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 145/282] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:31.905169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b77bf-9fea-45ff-89f2-57b75837079f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.905465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b77bf-9fea-45ff-89f2-57b75837079f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.905716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b77bf-9fea-45ff-89f2-57b75837079f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.927673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-016b77bf-9fea-45ff-89f2-57b75837079f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:31.928997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-016b77bf-9fea-45ff-89f2-57b75837079f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:31.954905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41504b51-8cba-4f03-af27-1801b7b7e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:31.955917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41504b51-8cba-4f03-af27-1801b7b7e976 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:31.956193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41504b51-8cba-4f03-af27-1801b7b7e976 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:31.957922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41504b51-8cba-4f03-af27-1801b7b7e976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:31.960181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41504b51-8cba-4f03-af27-1801b7b7e976 None admin] RBAC: Adding query filter params (role.id=c8d552ca72b140a8a815448527770138, user.id=20aa921d7fd344e399a8c3bb0e929355, scope.system=all) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:31.960566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41504b51-8cba-4f03-af27-1801b7b7e976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:31.962949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41504b51-8cba-4f03-af27-1801b7b7e976 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:31.994109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 138/283] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Oct 20 14:28:31 2022] GET /identity/v3/role_assignments?user.id=20aa921d7fd344e399a8c3bb0e929355&scope.system=all&role.id=c8d552ca72b140a8a815448527770138 => generated 510 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:32.024372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 146/284] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 20 14:28:32.394149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41504b51-8cba-4f03-af27-1801b7b7e976 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:32.406014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41504b51-8cba-4f03-af27-1801b7b7e976 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:32.413764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4270ca1-b75b-4cb3-be03-16b6d445c81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:32.414409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4270ca1-b75b-4cb3-be03-16b6d445c81f admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:32.414691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4270ca1-b75b-4cb3-be03-16b6d445c81f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:32.414958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4270ca1-b75b-4cb3-be03-16b6d445c81f admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:32.430159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4270ca1-b75b-4cb3-be03-16b6d445c81f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-e4270ca1-b75b-4cb3-be03-16b6d445c81f admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:32.435360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:32.436228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:32.437031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:32.437612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 139/285] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Oct 20 14:28:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:32.442320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b77bf-9fea-45ff-89f2-57b75837079f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:32.456000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b77bf-9fea-45ff-89f2-57b75837079f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:32.460628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c550e45-a2c9-4d69-971a-585aa8e8c533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:32.461234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c550e45-a2c9-4d69-971a-585aa8e8c533 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:32.461508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c550e45-a2c9-4d69-971a-585aa8e8c533 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:32.461841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c550e45-a2c9-4d69-971a-585aa8e8c533 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:32.462517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c550e45-a2c9-4d69-971a-585aa8e8c533 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:32.462820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c550e45-a2c9-4d69-971a-585aa8e8c533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:32.464474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c550e45-a2c9-4d69-971a-585aa8e8c533 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:32.476067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 147/286] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Oct 20 14:28:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:32.482770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4270ca1-b75b-4cb3-be03-16b6d445c81f admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:32.493369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4270ca1-b75b-4cb3-be03-16b6d445c81f admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:32.499055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc143ff-db7c-476e-be4d-ad9c606b90e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:32.499695 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc143ff-db7c-476e-be4d-ad9c606b90e8 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:32.499977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc143ff-db7c-476e-be4d-ad9c606b90e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:32.500251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc143ff-db7c-476e-be4d-ad9c606b90e8 admin admin] PATH_INFO: `/v3/groups/cef2035dbfdd43e496229c29189164c5` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:32.501430 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc143ff-db7c-476e-be4d-ad9c606b90e8 admin admin] RBAC: Authorizing `identity:get_group(group_id=cef2035dbfdd43e496229c29189164c5)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:32.503427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc143ff-db7c-476e-be4d-ad9c606b90e8 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:32.505043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 140/287] 10.222.0.13 () {56 vars in 1201 bytes} [Thu Oct 20 14:28:32 2022] GET /identity/v3/groups/cef2035dbfdd43e496229c29189164c5 => generated 226 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:32.510702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c550e45-a2c9-4d69-971a-585aa8e8c533 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:32.525642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c550e45-a2c9-4d69-971a-585aa8e8c533 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:32.530202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c50841-aa27-4c75-9f13-6fd69c0abe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:32.530838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50841-aa27-4c75-9f13-6fd69c0abe09 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:32.531120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50841-aa27-4c75-9f13-6fd69c0abe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:32.531395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50841-aa27-4c75-9f13-6fd69c0abe09 admin admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:32.533694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c50841-aa27-4c75-9f13-6fd69c0abe09 admin admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:32.535895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c50841-aa27-4c75-9f13-6fd69c0abe09 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:32.537578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 148/288] 10.222.0.13 () {56 vars in 1207 bytes} [Thu Oct 20 14:28:32 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:32.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc143ff-db7c-476e-be4d-ad9c606b90e8 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:32.555573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc143ff-db7c-476e-be4d-ad9c606b90e8 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:32.560811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa17e8b-6ccb-482b-8e97-0614025a2142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:32.561471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa17e8b-6ccb-482b-8e97-0614025a2142 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:32.561752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa17e8b-6ccb-482b-8e97-0614025a2142 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:32.562085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa17e8b-6ccb-482b-8e97-0614025a2142 admin admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346/groups/cef2035dbfdd43e496229c29189164c5/roles/dff349185bc94cb4a0e0f0c40425862f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:32.566606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa17e8b-6ccb-482b-8e97-0614025a2142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da9e929c8864aca8ed43b28bed3d346, group_id=cef2035dbfdd43e496229c29189164c5, role_id=dff349185bc94cb4a0e0f0c40425862f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:32.569400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa17e8b-6ccb-482b-8e97-0614025a2142 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:32.592426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 141/289] 10.222.0.13 () {58 vars in 1463 bytes} [Thu Oct 20 14:28:32 2022] PUT /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346/groups/cef2035dbfdd43e496229c29189164c5/roles/dff349185bc94cb4a0e0f0c40425862f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:28:32.812726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978f360-68ef-4d80-9f40-379c32c1b469 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:32.813014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978f360-68ef-4d80-9f40-379c32c1b469 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:32.813294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978f360-68ef-4d80-9f40-379c32c1b469 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:32.814980 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 149/290] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:32.820303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380ca5c-3b32-4996-8353-87caadf803b2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:32.820914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380ca5c-3b32-4996-8353-87caadf803b2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:32.820914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380ca5c-3b32-4996-8353-87caadf803b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:32.844643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-3380ca5c-3b32-4996-8353-87caadf803b2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:32.845853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3380ca5c-3b32-4996-8353-87caadf803b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:32.855246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:32.930632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 142/291] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 14:28:33.222258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978f360-68ef-4d80-9f40-379c32c1b469 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:33.235728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978f360-68ef-4d80-9f40-379c32c1b469 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:33.240176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee20a9-a39b-4229-9a1e-5e5953498908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:33.241720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee20a9-a39b-4229-9a1e-5e5953498908 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:33.242011 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee20a9-a39b-4229-9a1e-5e5953498908 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:33.242274 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee20a9-a39b-4229-9a1e-5e5953498908 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:33.244667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:33.255592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee20a9-a39b-4229-9a1e-5e5953498908 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-f6ee20a9-a39b-4229-9a1e-5e5953498908 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:33.261250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:33.262679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:33.263460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:33.269917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 150/292] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Oct 20 14:28:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:33.276155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380ca5c-3b32-4996-8353-87caadf803b2 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:33.287113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380ca5c-3b32-4996-8353-87caadf803b2 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:33.291572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26690c33-29fd-4f45-b74a-a289a43a8768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:33.292253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690c33-29fd-4f45-b74a-a289a43a8768 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:33.293125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690c33-29fd-4f45-b74a-a289a43a8768 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:33.293408 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690c33-29fd-4f45-b74a-a289a43a8768 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:33.294163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26690c33-29fd-4f45-b74a-a289a43a8768 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:33.294470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26690c33-29fd-4f45-b74a-a289a43a8768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:33.296108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26690c33-29fd-4f45-b74a-a289a43a8768 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:33.308752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 143/293] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Oct 20 14:28:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:33.314444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee20a9-a39b-4229-9a1e-5e5953498908 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:33.329654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee20a9-a39b-4229-9a1e-5e5953498908 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:33.334326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2051dce-b0be-49de-af3e-12686805cd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:33.334964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051dce-b0be-49de-af3e-12686805cd9b admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:33.335247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051dce-b0be-49de-af3e-12686805cd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:33.335530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051dce-b0be-49de-af3e-12686805cd9b admin admin] PATH_INFO: `/v3/groups/0b0f7b9346124a9f9f1892140cfbd0b3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:33.341079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2051dce-b0be-49de-af3e-12686805cd9b admin admin] RBAC: Authorizing `identity:get_group(group_id=0b0f7b9346124a9f9f1892140cfbd0b3)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:33.343085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2051dce-b0be-49de-af3e-12686805cd9b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:33.344791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 151/294] 10.222.0.13 () {56 vars in 1201 bytes} [Thu Oct 20 14:28:33 2022] GET /identity/v3/groups/0b0f7b9346124a9f9f1892140cfbd0b3 => generated 229 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:33.351909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26690c33-29fd-4f45-b74a-a289a43a8768 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:33.362819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26690c33-29fd-4f45-b74a-a289a43a8768 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:33.367364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7d43a-7f28-419b-9bf8-21b2900165eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:33.368258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d43a-7f28-419b-9bf8-21b2900165eb admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:33.368598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d43a-7f28-419b-9bf8-21b2900165eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:33.368874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d43a-7f28-419b-9bf8-21b2900165eb admin admin] PATH_INFO: `/v3/projects/cfa845f715934496950bc728f50bbff3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:33.369998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7d43a-7f28-419b-9bf8-21b2900165eb admin admin] RBAC: Authorizing `identity:get_project(project_id=cfa845f715934496950bc728f50bbff3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:33.371861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7d43a-7f28-419b-9bf8-21b2900165eb admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:33.374425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 144/295] 10.222.0.13 () {56 vars in 1207 bytes} [Thu Oct 20 14:28:33 2022] GET /identity/v3/projects/cfa845f715934496950bc728f50bbff3 => generated 344 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:33.380018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2051dce-b0be-49de-af3e-12686805cd9b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:33.391016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2051dce-b0be-49de-af3e-12686805cd9b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:33.395673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa99c8-6374-4b09-9c40-d27383b3047b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:33.396400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa99c8-6374-4b09-9c40-d27383b3047b admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:33.396693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa99c8-6374-4b09-9c40-d27383b3047b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:33.396973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa99c8-6374-4b09-9c40-d27383b3047b admin admin] PATH_INFO: `/v3/projects/cfa845f715934496950bc728f50bbff3/groups/0b0f7b9346124a9f9f1892140cfbd0b3/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:33.398988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa99c8-6374-4b09-9c40-d27383b3047b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa845f715934496950bc728f50bbff3, group_id=0b0f7b9346124a9f9f1892140cfbd0b3, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:33.401845 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa99c8-6374-4b09-9c40-d27383b3047b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:33.418874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 152/296] 10.222.0.13 () {58 vars in 1463 bytes} [Thu Oct 20 14:28:33 2022] PUT /identity/v3/projects/cfa845f715934496950bc728f50bbff3/groups/0b0f7b9346124a9f9f1892140cfbd0b3/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:33.641596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a1ea3-96e0-4e47-9d3e-4bf536709ca5 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:33.641934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a1ea3-96e0-4e47-9d3e-4bf536709ca5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:33.642131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a1ea3-96e0-4e47-9d3e-4bf536709ca5 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:33.643570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 145/297] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:33.649198 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4528c99-dd71-495a-ab3f-95b03c79ae1e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:33.649636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4528c99-dd71-495a-ab3f-95b03c79ae1e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:33.649803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4528c99-dd71-495a-ab3f-95b03c79ae1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:33.674204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f4528c99-dd71-495a-ab3f-95b03c79ae1e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:33.674204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f4528c99-dd71-495a-ab3f-95b03c79ae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:33.756614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 153/298] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:34.214619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a1ea3-96e0-4e47-9d3e-4bf536709ca5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:34.324268 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a1ea3-96e0-4e47-9d3e-4bf536709ca5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:34.363084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148ea22-f11c-4802-b70d-7e0db9ed00a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:34.363721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148ea22-f11c-4802-b70d-7e0db9ed00a9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:34.363999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148ea22-f11c-4802-b70d-7e0db9ed00a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:34.364272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148ea22-f11c-4802-b70d-7e0db9ed00a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:34.373189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:34.376859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148ea22-f11c-4802-b70d-7e0db9ed00a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-7148ea22-f11c-4802-b70d-7e0db9ed00a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:34.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:34.382786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:34.383567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:34.387928 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 146/299] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:34.389342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4528c99-dd71-495a-ab3f-95b03c79ae1e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:34.511342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4528c99-dd71-495a-ab3f-95b03c79ae1e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:34.514042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:34.560327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4524f-6cd1-4229-a43e-71befbac47c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:34.560990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4524f-6cd1-4229-a43e-71befbac47c5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:34.561270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4524f-6cd1-4229-a43e-71befbac47c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:34.561546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4524f-6cd1-4229-a43e-71befbac47c5 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:34.562235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4524f-6cd1-4229-a43e-71befbac47c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:34.562539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4524f-6cd1-4229-a43e-71befbac47c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:34.563981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4524f-6cd1-4229-a43e-71befbac47c5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:34.569136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c570a-7778-4d80-ae3f-f239a6f7aa5a None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:34.569757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c570a-7778-4d80-ae3f-f239a6f7aa5a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:34.570261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c570a-7778-4d80-ae3f-f239a6f7aa5a None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:34.572510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 147/300] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:34.575996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 154/301] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:34.579036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e85b3-9f72-41c1-8048-b5f182a0254b None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:34.579603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e85b3-9f72-41c1-8048-b5f182a0254b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:34.580111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e85b3-9f72-41c1-8048-b5f182a0254b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:34.582481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4524f-6cd1-4229-a43e-71befbac47c5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:34.613496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ed0e85b3-9f72-41c1-8048-b5f182a0254b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:34.614682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ed0e85b3-9f72-41c1-8048-b5f182a0254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:34.693463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 148/302] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:34.702932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efcb0c-ebe6-45ed-acb0-a42ab80b6f17 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:34.703470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efcb0c-ebe6-45ed-acb0-a42ab80b6f17 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:34.703769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efcb0c-ebe6-45ed-acb0-a42ab80b6f17 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:34.706108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 149/303] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:34.711032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6fe5b-8e4f-4c96-b57d-3e3ad0e668f2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:34.711312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6fe5b-8e4f-4c96-b57d-3e3ad0e668f2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:34.711580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6fe5b-8e4f-4c96-b57d-3e3ad0e668f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:34.725344 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4524f-6cd1-4229-a43e-71befbac47c5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:34.732130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-54a6fe5b-8e4f-4c96-b57d-3e3ad0e668f2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:34.733502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-54a6fe5b-8e4f-4c96-b57d-3e3ad0e668f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:34.789595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc841af-9cb9-42d0-aab2-e798eaafd7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:34.790125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc841af-9cb9-42d0-aab2-e798eaafd7a7 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:34.790406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc841af-9cb9-42d0-aab2-e798eaafd7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:34.790682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc841af-9cb9-42d0-aab2-e798eaafd7a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:34.862046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:34.877126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 150/304] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:34.907788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc841af-9cb9-42d0-aab2-e798eaafd7a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:34.909297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc841af-9cb9-42d0-aab2-e798eaafd7a7 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:35.016325 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6fe5b-8e4f-4c96-b57d-3e3ad0e668f2 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:35.029558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 155/305] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 20 14:28:35.036076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc841af-9cb9-42d0-aab2-e798eaafd7a7 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:35.112794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:35.156712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:35.192365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6fe5b-8e4f-4c96-b57d-3e3ad0e668f2 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:35.226578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc841af-9cb9-42d0-aab2-e798eaafd7a7 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:35.244835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefde4e-07ae-40e7-943e-e6586fec4e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:35.245481 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefde4e-07ae-40e7-943e-e6586fec4e2f None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:35.246350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefde4e-07ae-40e7-943e-e6586fec4e2f None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:35.246817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefde4e-07ae-40e7-943e-e6586fec4e2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:35.269964 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebefde4e-07ae-40e7-943e-e6586fec4e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-ebefde4e-07ae-40e7-943e-e6586fec4e2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:35.274907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:35.275753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:35.276552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:35.277682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 151/306] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:28:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:35.285795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefde4e-07ae-40e7-943e-e6586fec4e2f None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:35.313795 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b4b97-4bcd-48f3-b653-20ef214a8c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:35.314433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b4b97-4bcd-48f3-b653-20ef214a8c89 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:35.314716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b4b97-4bcd-48f3-b653-20ef214a8c89 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:35.314992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b4b97-4bcd-48f3-b653-20ef214a8c89 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:35.316129 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b4b97-4bcd-48f3-b653-20ef214a8c89 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:35.317604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b4b97-4bcd-48f3-b653-20ef214a8c89 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:35.324024 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 156/307] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:35 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:35.324024 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b4b97-4bcd-48f3-b653-20ef214a8c89 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:35.433346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefde4e-07ae-40e7-943e-e6586fec4e2f None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:35.471162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadafe7b-1e5d-41b1-8822-8fde2461dd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:35.471943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadafe7b-1e5d-41b1-8822-8fde2461dd16 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:35.472317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadafe7b-1e5d-41b1-8822-8fde2461dd16 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:35.472715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadafe7b-1e5d-41b1-8822-8fde2461dd16 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:35.473480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadafe7b-1e5d-41b1-8822-8fde2461dd16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:35.473879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadafe7b-1e5d-41b1-8822-8fde2461dd16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:35.475365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadafe7b-1e5d-41b1-8822-8fde2461dd16 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:35.478301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b4b97-4bcd-48f3-b653-20ef214a8c89 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:35.481602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:35.486452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 152/308] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:35.487510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadafe7b-1e5d-41b1-8822-8fde2461dd16 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:35.519065 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016335e0-b6b3-4e00-8b05-658fc6cfa29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:35.519704 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016335e0-b6b3-4e00-8b05-658fc6cfa29e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:35.519988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016335e0-b6b3-4e00-8b05-658fc6cfa29e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:35.520267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016335e0-b6b3-4e00-8b05-658fc6cfa29e None admin] PATH_INFO: `/v3/roles/member` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:35.522953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:35.537514 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016335e0-b6b3-4e00-8b05-658fc6cfa29e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-016335e0-b6b3-4e00-8b05-658fc6cfa29e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:35.542977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:35.543956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 157/309] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:35.545710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016335e0-b6b3-4e00-8b05-658fc6cfa29e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:35.636911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadafe7b-1e5d-41b1-8822-8fde2461dd16 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:35.692059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ba68-2c0a-4aab-b5c2-47397cf9d218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:35.693278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ba68-2c0a-4aab-b5c2-47397cf9d218 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:35.693740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ba68-2c0a-4aab-b5c2-47397cf9d218 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:35.694197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ba68-2c0a-4aab-b5c2-47397cf9d218 None admin] PATH_INFO: `/v3/groups/cef2035dbfdd43e496229c29189164c5` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:35.695525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7ba68-2c0a-4aab-b5c2-47397cf9d218 None admin] RBAC: Authorizing `identity:get_group(group_id=cef2035dbfdd43e496229c29189164c5)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:35.696766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016335e0-b6b3-4e00-8b05-658fc6cfa29e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:35.697449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7ba68-2c0a-4aab-b5c2-47397cf9d218 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:35.701313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 153/310] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Oct 20 14:28:35 2022] GET /identity/v3/groups/cef2035dbfdd43e496229c29189164c5 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:35.702367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ba68-2c0a-4aab-b5c2-47397cf9d218 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:35.739613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8f3cf-a3a4-46fc-a6f4-987b62b61dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:35.740157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8f3cf-a3a4-46fc-a6f4-987b62b61dcb None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:35.740321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8f3cf-a3a4-46fc-a6f4-987b62b61dcb None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:35.740642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8f3cf-a3a4-46fc-a6f4-987b62b61dcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:35.770389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:35.820659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:35.828137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8f3cf-a3a4-46fc-a6f4-987b62b61dcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:35.831829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8f3cf-a3a4-46fc-a6f4-987b62b61dcb None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:35.838817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ba68-2c0a-4aab-b5c2-47397cf9d218 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:35.890980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f40e3-5485-4e5b-9a7b-67776a9725a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:35.891759 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f40e3-5485-4e5b-9a7b-67776a9725a8 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:35.892047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f40e3-5485-4e5b-9a7b-67776a9725a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:35.892615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f40e3-5485-4e5b-9a7b-67776a9725a8 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:35.893163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f40e3-5485-4e5b-9a7b-67776a9725a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:35.893472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f40e3-5485-4e5b-9a7b-67776a9725a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:35.894871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f40e3-5485-4e5b-9a7b-67776a9725a8 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:35.905722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 154/311] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:28:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:35.906578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f40e3-5485-4e5b-9a7b-67776a9725a8 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:35.975909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 158/312] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:35.977420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267b7cb-4562-48b8-8e5a-78a74e01d415 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:35.977667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267b7cb-4562-48b8-8e5a-78a74e01d415 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:35.977950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267b7cb-4562-48b8-8e5a-78a74e01d415 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:35.979192 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 159/313] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:35.980492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267b7cb-4562-48b8-8e5a-78a74e01d415 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:36.039462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f40e3-5485-4e5b-9a7b-67776a9725a8 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:36.054486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:36.083756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f16a4-0ec7-4e29-814d-3834bb4c541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:36.084408 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f16a4-0ec7-4e29-814d-3834bb4c541d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.084696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f16a4-0ec7-4e29-814d-3834bb4c541d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.084972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f16a4-0ec7-4e29-814d-3834bb4c541d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.085707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f16a4-0ec7-4e29-814d-3834bb4c541d None admin] RBAC: Adding query filter params (group.id=cef2035dbfdd43e496229c29189164c5, role.id=a73215dcd05645c89a2f742200bbb0c9, scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:36.086019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f16a4-0ec7-4e29-814d-3834bb4c541d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:36.088173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f16a4-0ec7-4e29-814d-3834bb4c541d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:36.094569 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 155/314] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Oct 20 14:28:35 2022] GET /identity/v3/role_assignments?group.id=cef2035dbfdd43e496229c29189164c5&scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81&role.id=a73215dcd05645c89a2f742200bbb0c9 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:36.095379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f16a4-0ec7-4e29-814d-3834bb4c541d None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:36.107027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:36.123885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267b7cb-4562-48b8-8e5a-78a74e01d415 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:36.178502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23681b-a85f-4df6-b032-081089fddbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:36.179355 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23681b-a85f-4df6-b032-081089fddbc5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.179739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23681b-a85f-4df6-b032-081089fddbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.180117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23681b-a85f-4df6-b032-081089fddbc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.238499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f16a4-0ec7-4e29-814d-3834bb4c541d None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:36.264913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23681b-a85f-4df6-b032-081089fddbc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:36.266345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23681b-a85f-4df6-b032-081089fddbc5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:36.281039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08150137-e1c1-4406-8404-f31099827409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:36.282363 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08150137-e1c1-4406-8404-f31099827409 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.282786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08150137-e1c1-4406-8404-f31099827409 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.283210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08150137-e1c1-4406-8404-f31099827409 None admin] PATH_INFO: `/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.284677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08150137-e1c1-4406-8404-f31099827409 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dde0c3629144eabb2ec9e6e6c16a81)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:36.286471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08150137-e1c1-4406-8404-f31099827409 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:36.288857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 156/315] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:36 2022] GET /identity/v3/projects/e8dde0c3629144eabb2ec9e6e6c16a81 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:36.290355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b55a3-2e7e-431a-a5a6-4059ec5d3fc0 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.290950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b55a3-2e7e-431a-a5a6-4059ec5d3fc0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.293598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b55a3-2e7e-431a-a5a6-4059ec5d3fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.315804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d21b55a3-2e7e-431a-a5a6-4059ec5d3fc0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:36.317392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d21b55a3-2e7e-431a-a5a6-4059ec5d3fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:36.383103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:36.387407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:36.396661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 160/316] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:36.397490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23681b-a85f-4df6-b032-081089fddbc5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:36.416399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 157/317] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:36.418258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97819af4-abee-4259-ab8c-cf05b56cfbb0 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.418523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97819af4-abee-4259-ab8c-cf05b56cfbb0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.418792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97819af4-abee-4259-ab8c-cf05b56cfbb0 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.420035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 158/318] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:36.420778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97819af4-abee-4259-ab8c-cf05b56cfbb0 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:36.535213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23681b-a85f-4df6-b032-081089fddbc5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:36.551798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97819af4-abee-4259-ab8c-cf05b56cfbb0 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:36.580532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36706a1a-79a1-4727-8ec5-16d9e98c9c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:36.581172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36706a1a-79a1-4727-8ec5-16d9e98c9c40 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.581447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36706a1a-79a1-4727-8ec5-16d9e98c9c40 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.582030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36706a1a-79a1-4727-8ec5-16d9e98c9c40 None admin] PATH_INFO: `/v3/groups/cef2035dbfdd43e496229c29189164c5` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.582843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36706a1a-79a1-4727-8ec5-16d9e98c9c40 None admin] RBAC: Authorizing `identity:get_group(group_id=cef2035dbfdd43e496229c29189164c5)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:36.584278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36706a1a-79a1-4727-8ec5-16d9e98c9c40 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:36.586173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 161/319] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Oct 20 14:28:36 2022] GET /identity/v3/groups/cef2035dbfdd43e496229c29189164c5 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:36.587425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45b9ef-24ab-40b8-bdb8-0d574aa16eef None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.587425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45b9ef-24ab-40b8-bdb8-0d574aa16eef None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.587596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45b9ef-24ab-40b8-bdb8-0d574aa16eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.599418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790a956-dfd2-49fe-9a37-4aaf94bd9958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:36.600051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a956-dfd2-49fe-9a37-4aaf94bd9958 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.600324 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a956-dfd2-49fe-9a37-4aaf94bd9958 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.600636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a956-dfd2-49fe-9a37-4aaf94bd9958 None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.601743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790a956-dfd2-49fe-9a37-4aaf94bd9958 None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:36.603180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790a956-dfd2-49fe-9a37-4aaf94bd9958 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:36.605002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 159/320] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:36 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:36.605881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790a956-dfd2-49fe-9a37-4aaf94bd9958 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:36.613233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-bf45b9ef-24ab-40b8-bdb8-0d574aa16eef None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:36.615083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bf45b9ef-24ab-40b8-bdb8-0d574aa16eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:36.660419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:36.687725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:36.702923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 162/321] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:36.708366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45b9ef-24ab-40b8-bdb8-0d574aa16eef None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:36.732025 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790a956-dfd2-49fe-9a37-4aaf94bd9958 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:36.774714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2f80-5a39-4681-b960-32b4af738d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:36.774714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2f80-5a39-4681-b960-32b4af738d22 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.774714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2f80-5a39-4681-b960-32b4af738d22 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.775086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2f80-5a39-4681-b960-32b4af738d22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.775610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c2f80-5a39-4681-b960-32b4af738d22 None admin] RBAC: Adding query filter params (group.id=cef2035dbfdd43e496229c29189164c5, role.id=dff349185bc94cb4a0e0f0c40425862f, scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:36.775922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c2f80-5a39-4681-b960-32b4af738d22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:36.777932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c2f80-5a39-4681-b960-32b4af738d22 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:36.784742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 160/322] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Oct 20 14:28:36 2022] GET /identity/v3/role_assignments?group.id=cef2035dbfdd43e496229c29189164c5&scope.project.id=e8dde0c3629144eabb2ec9e6e6c16a81&role.id=dff349185bc94cb4a0e0f0c40425862f => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:36.785614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2f80-5a39-4681-b960-32b4af738d22 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:36.850884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45b9ef-24ab-40b8-bdb8-0d574aa16eef None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:36.897433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270448e-b6a8-466c-8583-b8e057a3bc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:36.898088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270448e-b6a8-466c-8583-b8e057a3bc27 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.898367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270448e-b6a8-466c-8583-b8e057a3bc27 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.898701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270448e-b6a8-466c-8583-b8e057a3bc27 None admin] PATH_INFO: `/v3/groups/cef2035dbfdd43e496229c29189164c5` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.899879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270448e-b6a8-466c-8583-b8e057a3bc27 None admin] RBAC: Authorizing `identity:get_group(group_id=cef2035dbfdd43e496229c29189164c5)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:36.902467 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270448e-b6a8-466c-8583-b8e057a3bc27 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:36.904621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 163/323] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Oct 20 14:28:36 2022] GET /identity/v3/groups/cef2035dbfdd43e496229c29189164c5 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:36.911380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270448e-b6a8-466c-8583-b8e057a3bc27 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:36.916592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2f80-5a39-4681-b960-32b4af738d22 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:36.932373 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:36.964169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feacc20-2ad5-4ba2-bc59-d255ad01a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:36.965901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feacc20-2ad5-4ba2-bc59-d255ad01a088 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:36.966368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feacc20-2ad5-4ba2-bc59-d255ad01a088 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:36.966822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feacc20-2ad5-4ba2-bc59-d255ad01a088 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:36.982585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feacc20-2ad5-4ba2-bc59-d255ad01a088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-4feacc20-2ad5-4ba2-bc59-d255ad01a088 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:36.988337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:36.989270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:36.990115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:36.990734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:36.993093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 161/324] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:28:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:36.996980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feacc20-2ad5-4ba2-bc59-d255ad01a088 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:37.048561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270448e-b6a8-466c-8583-b8e057a3bc27 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:37.092199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44df20-86d8-49e2-87e6-95d482ef29d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:37.092923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44df20-86d8-49e2-87e6-95d482ef29d4 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:37.093199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44df20-86d8-49e2-87e6-95d482ef29d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:37.093469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44df20-86d8-49e2-87e6-95d482ef29d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:37.094372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44df20-86d8-49e2-87e6-95d482ef29d4 None admin] RBAC: Adding query filter params (group.id=cef2035dbfdd43e496229c29189164c5, role.id=a73215dcd05645c89a2f742200bbb0c9, scope.project.id=6da9e929c8864aca8ed43b28bed3d346) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:37.094685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44df20-86d8-49e2-87e6-95d482ef29d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:37.098659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44df20-86d8-49e2-87e6-95d482ef29d4 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:37.103949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 164/325] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Oct 20 14:28:36 2022] GET /identity/v3/role_assignments?group.id=cef2035dbfdd43e496229c29189164c5&scope.project.id=6da9e929c8864aca8ed43b28bed3d346&role.id=a73215dcd05645c89a2f742200bbb0c9 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:37.105935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44df20-86d8-49e2-87e6-95d482ef29d4 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:37.137206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feacc20-2ad5-4ba2-bc59-d255ad01a088 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:37.181210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0220b31-49d2-42e1-9372-990ee103f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:37.181866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0220b31-49d2-42e1-9372-990ee103f2e9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:37.182144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0220b31-49d2-42e1-9372-990ee103f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:37.182529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0220b31-49d2-42e1-9372-990ee103f2e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:37.197189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0220b31-49d2-42e1-9372-990ee103f2e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-f0220b31-49d2-42e1-9372-990ee103f2e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:37.203326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:37.204241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:37.205061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 162/326] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:28:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:37.211815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0220b31-49d2-42e1-9372-990ee103f2e9 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:37.247034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:37.251707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44df20-86d8-49e2-87e6-95d482ef29d4 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:37.285282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:37.301597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f608-1df3-4d8e-bd08-243ca2b217e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:37.302237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f608-1df3-4d8e-bd08-243ca2b217e8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:37.302590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f608-1df3-4d8e-bd08-243ca2b217e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:37.302876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f608-1df3-4d8e-bd08-243ca2b217e8 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:37.303539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32f608-1df3-4d8e-bd08-243ca2b217e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:37.303851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32f608-1df3-4d8e-bd08-243ca2b217e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:37.305261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32f608-1df3-4d8e-bd08-243ca2b217e8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:37.316690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 165/327] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:37.322352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f608-1df3-4d8e-bd08-243ca2b217e8 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:37.345131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0220b31-49d2-42e1-9372-990ee103f2e9 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:37.385626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e4d5f-e2ea-4900-a87a-37b4d3177c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:37.387858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e4d5f-e2ea-4900-a87a-37b4d3177c8c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:37.387858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e4d5f-e2ea-4900-a87a-37b4d3177c8c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:37.387858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e4d5f-e2ea-4900-a87a-37b4d3177c8c None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:37.388171 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e4d5f-e2ea-4900-a87a-37b4d3177c8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:37.388753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e4d5f-e2ea-4900-a87a-37b4d3177c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:37.389861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e4d5f-e2ea-4900-a87a-37b4d3177c8c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:37.398696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 163/328] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:37.405524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e4d5f-e2ea-4900-a87a-37b4d3177c8c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:37.459738 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f608-1df3-4d8e-bd08-243ca2b217e8 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:37.501701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c2110-d4a2-413c-a45e-fb51acd507d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:37.502699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c2110-d4a2-413c-a45e-fb51acd507d1 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:37.503127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c2110-d4a2-413c-a45e-fb51acd507d1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:37.503407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c2110-d4a2-413c-a45e-fb51acd507d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:37.510653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:37.525460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e4d5f-e2ea-4900-a87a-37b4d3177c8c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:37.567785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:37.569730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad07ac1-2c8c-4ada-8549-d96786eda502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:37.570387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad07ac1-2c8c-4ada-8549-d96786eda502 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:37.570675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad07ac1-2c8c-4ada-8549-d96786eda502 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:37.571090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad07ac1-2c8c-4ada-8549-d96786eda502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:37.583491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c2110-d4a2-413c-a45e-fb51acd507d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:37.584721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c2110-d4a2-413c-a45e-fb51acd507d1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:37.648585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad07ac1-2c8c-4ada-8549-d96786eda502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:37.650072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad07ac1-2c8c-4ada-8549-d96786eda502 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:37.693782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 166/329] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 20 14:28:37.699352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c2110-d4a2-413c-a45e-fb51acd507d1 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:37.736906 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:37.750540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 164/330] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 20 14:28:37.756407 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad07ac1-2c8c-4ada-8549-d96786eda502 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:37.781007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:37.796736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c2110-d4a2-413c-a45e-fb51acd507d1 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:37.829759 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e559b9-229f-4675-8393-534c7e405ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:37.830317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e559b9-229f-4675-8393-534c7e405ffd None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:37.830597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e559b9-229f-4675-8393-534c7e405ffd None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:37.830878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e559b9-229f-4675-8393-534c7e405ffd None admin] PATH_INFO: `/v3/projects/6da9e929c8864aca8ed43b28bed3d346` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:37.831986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e559b9-229f-4675-8393-534c7e405ffd None admin] RBAC: Authorizing `identity:get_project(project_id=6da9e929c8864aca8ed43b28bed3d346)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:37.833478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e559b9-229f-4675-8393-534c7e405ffd None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:37.835249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 167/331] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:37 2022] GET /identity/v3/projects/6da9e929c8864aca8ed43b28bed3d346 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:37.840632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e559b9-229f-4675-8393-534c7e405ffd None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:37.856831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad07ac1-2c8c-4ada-8549-d96786eda502 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:37.890389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3eb2e5-d60a-48ff-b0b3-c2f82efe45ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:37.891015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3eb2e5-d60a-48ff-b0b3-c2f82efe45ae None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:37.891891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3eb2e5-d60a-48ff-b0b3-c2f82efe45ae None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:37.891891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3eb2e5-d60a-48ff-b0b3-c2f82efe45ae None admin] PATH_INFO: `/v3/projects/cfa845f715934496950bc728f50bbff3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:37.892813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3eb2e5-d60a-48ff-b0b3-c2f82efe45ae None admin] RBAC: Authorizing `identity:get_project(project_id=cfa845f715934496950bc728f50bbff3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:37.894274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3eb2e5-d60a-48ff-b0b3-c2f82efe45ae None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:37.896064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 165/332] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:28:37 2022] GET /identity/v3/projects/cfa845f715934496950bc728f50bbff3 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:37.901465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3eb2e5-d60a-48ff-b0b3-c2f82efe45ae None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:37.939400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e559b9-229f-4675-8393-534c7e405ffd None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:37.956891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:37.971888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91002691-1194-4640-aaf1-ce8d5bd6a98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:37.972549 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91002691-1194-4640-aaf1-ce8d5bd6a98e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:37.972820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91002691-1194-4640-aaf1-ce8d5bd6a98e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:37.973088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91002691-1194-4640-aaf1-ce8d5bd6a98e None admin] PATH_INFO: `/v3/groups/cef2035dbfdd43e496229c29189164c5` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:37.974166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91002691-1194-4640-aaf1-ce8d5bd6a98e None admin] RBAC: Authorizing `identity:get_group(group_id=cef2035dbfdd43e496229c29189164c5)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:37.975603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91002691-1194-4640-aaf1-ce8d5bd6a98e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:37.977173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 168/333] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Oct 20 14:28:37 2022] GET /identity/v3/groups/cef2035dbfdd43e496229c29189164c5 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:37.983643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91002691-1194-4640-aaf1-ce8d5bd6a98e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:38.001793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3eb2e5-d60a-48ff-b0b3-c2f82efe45ae None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:38.006996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:38.034766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb30713-a607-4c51-81de-c1a1e47c2005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:38.035383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb30713-a607-4c51-81de-c1a1e47c2005 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:38.035658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb30713-a607-4c51-81de-c1a1e47c2005 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:38.035923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb30713-a607-4c51-81de-c1a1e47c2005 None admin] PATH_INFO: `/v3/groups/0b0f7b9346124a9f9f1892140cfbd0b3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:38.037041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb30713-a607-4c51-81de-c1a1e47c2005 None admin] RBAC: Authorizing `identity:get_group(group_id=0b0f7b9346124a9f9f1892140cfbd0b3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:38.038482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb30713-a607-4c51-81de-c1a1e47c2005 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:38.040041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 166/334] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Oct 20 14:28:37 2022] GET /identity/v3/groups/0b0f7b9346124a9f9f1892140cfbd0b3 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:38.045912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb30713-a607-4c51-81de-c1a1e47c2005 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:38.080741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91002691-1194-4640-aaf1-ce8d5bd6a98e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:38.113729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72f725-36d4-4253-a860-f0faca460c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:38.114323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72f725-36d4-4253-a860-f0faca460c41 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:38.114598 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72f725-36d4-4253-a860-f0faca460c41 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:38.114872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72f725-36d4-4253-a860-f0faca460c41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:38.115584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72f725-36d4-4253-a860-f0faca460c41 None admin] RBAC: Adding query filter params (group.id=cef2035dbfdd43e496229c29189164c5, role.id=dff349185bc94cb4a0e0f0c40425862f, scope.project.id=6da9e929c8864aca8ed43b28bed3d346) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:38.115886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72f725-36d4-4253-a860-f0faca460c41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:38.117320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72f725-36d4-4253-a860-f0faca460c41 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:38.124231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 169/335] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Oct 20 14:28:37 2022] GET /identity/v3/role_assignments?group.id=cef2035dbfdd43e496229c29189164c5&scope.project.id=6da9e929c8864aca8ed43b28bed3d346&role.id=dff349185bc94cb4a0e0f0c40425862f => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:38.146174 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb30713-a607-4c51-81de-c1a1e47c2005 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:38.171427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:38.182740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6004929-5f1c-41a6-b0f1-07e44a8ab406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:38.184406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6004929-5f1c-41a6-b0f1-07e44a8ab406 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:38.184700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6004929-5f1c-41a6-b0f1-07e44a8ab406 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:38.184969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6004929-5f1c-41a6-b0f1-07e44a8ab406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:38.185715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6004929-5f1c-41a6-b0f1-07e44a8ab406 None admin] RBAC: Adding query filter params (group.id=0b0f7b9346124a9f9f1892140cfbd0b3, role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.project.id=cfa845f715934496950bc728f50bbff3) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:38.186021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6004929-5f1c-41a6-b0f1-07e44a8ab406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:38.187498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6004929-5f1c-41a6-b0f1-07e44a8ab406 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:38.193908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 167/336] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Oct 20 14:28:38 2022] GET /identity/v3/role_assignments?group.id=0b0f7b9346124a9f9f1892140cfbd0b3&scope.project.id=cfa845f715934496950bc728f50bbff3&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:28:40.969619 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db34c6d-cc4d-459c-80e7-87d3f9fa84a1 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:40.970649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db34c6d-cc4d-459c-80e7-87d3f9fa84a1 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:40.971134 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db34c6d-cc4d-459c-80e7-87d3f9fa84a1 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:40.972736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 170/337] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:40.978034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae75ed-3b31-4cd6-806f-de73c51df6c5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:40.978853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae75ed-3b31-4cd6-806f-de73c51df6c5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:40.979314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae75ed-3b31-4cd6-806f-de73c51df6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:41.005317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-60ae75ed-3b31-4cd6-806f-de73c51df6c5 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:41.006806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-60ae75ed-3b31-4cd6-806f-de73c51df6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:41.086576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 168/338] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:41.360544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edf776-da40-4ce8-9055-d9f8415e9ebd None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:41.360834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edf776-da40-4ce8-9055-d9f8415e9ebd None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:41.361112 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edf776-da40-4ce8-9055-d9f8415e9ebd None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:41.362702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 171/339] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:41.368432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95156bfe-5cd6-49d9-a8a6-4713199a0a4b None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:41.368724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95156bfe-5cd6-49d9-a8a6-4713199a0a4b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:41.369000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95156bfe-5cd6-49d9-a8a6-4713199a0a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:41.393353 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-95156bfe-5cd6-49d9-a8a6-4713199a0a4b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:41.394542 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-95156bfe-5cd6-49d9-a8a6-4713199a0a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:41.397711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0d68d-caec-4701-959a-077848428535 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:41.397993 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0d68d-caec-4701-959a-077848428535 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:41.398266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0d68d-caec-4701-959a-077848428535 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:41.399844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 172/340] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:41.409846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28ac2c-a13c-42ae-8f18-ce9aec899ad1 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:41.410125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28ac2c-a13c-42ae-8f18-ce9aec899ad1 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:41.410624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28ac2c-a13c-42ae-8f18-ce9aec899ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:41.432909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-bc28ac2c-a13c-42ae-8f18-ce9aec899ad1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:41.434533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bc28ac2c-a13c-42ae-8f18-ce9aec899ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:41.480393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 169/341] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:41.480393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95156bfe-5cd6-49d9-a8a6-4713199a0a4b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:41.533946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 173/342] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:41.568709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:41.641055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95156bfe-5cd6-49d9-a8a6-4713199a0a4b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:41.649879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ebef5-6bd9-495a-8a58-0f4c19517530 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:41.650170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ebef5-6bd9-495a-8a58-0f4c19517530 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:41.650451 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ebef5-6bd9-495a-8a58-0f4c19517530 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:41.652208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 174/343] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:41.657532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298589-1c8a-40a0-af87-d9738398d600 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:41.657846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298589-1c8a-40a0-af87-d9738398d600 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:41.658125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298589-1c8a-40a0-af87-d9738398d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:41.690984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e2298589-1c8a-40a0-af87-d9738398d600 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:41.692164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e2298589-1c8a-40a0-af87-d9738398d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:41.695447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1b365-edb5-4022-a6f5-2a11033479e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:41.696088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1b365-edb5-4022-a6f5-2a11033479e5 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:41.696383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1b365-edb5-4022-a6f5-2a11033479e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:41.696658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1b365-edb5-4022-a6f5-2a11033479e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:41.715854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1b365-edb5-4022-a6f5-2a11033479e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:41.717352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1b365-edb5-4022-a6f5-2a11033479e5 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:41.719577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-0ec1b365-edb5-4022-a6f5-2a11033479e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:41.746322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:41.747314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:41.748167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:41.748167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:41.748167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:41.748167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 170/344] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:41.757262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1b365-edb5-4022-a6f5-2a11033479e5 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:41.799216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 175/345] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:41.835659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2298589-1c8a-40a0-af87-d9738398d600 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:41.928648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:41.932376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1b365-edb5-4022-a6f5-2a11033479e5 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:41.971894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a2452-cbae-4788-98cd-3a89c4ae89c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:41.972954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a2452-cbae-4788-98cd-3a89c4ae89c4 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:41.973495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a2452-cbae-4788-98cd-3a89c4ae89c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:41.973990 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a2452-cbae-4788-98cd-3a89c4ae89c4 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:41.974838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a2452-cbae-4788-98cd-3a89c4ae89c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:41.975413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a2452-cbae-4788-98cd-3a89c4ae89c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:41.977128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a2452-cbae-4788-98cd-3a89c4ae89c4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:41.979874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2298589-1c8a-40a0-af87-d9738398d600 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:41.990282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 171/346] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:41.991867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a2452-cbae-4788-98cd-3a89c4ae89c4 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:42.020926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabaa04-cadf-4ac3-b34c-4a945b50e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.021805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabaa04-cadf-4ac3-b34c-4a945b50e4e6 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.022229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabaa04-cadf-4ac3-b34c-4a945b50e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.022887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabaa04-cadf-4ac3-b34c-4a945b50e4e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.036124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabaa04-cadf-4ac3-b34c-4a945b50e4e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.038242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabaa04-cadf-4ac3-b34c-4a945b50e4e6 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.047543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-1dabaa04-cadf-4ac3-b34c-4a945b50e4e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:42.059864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:42.061016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:42.062053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:42.062053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:42.062053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:42.068668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 176/347] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:42.070234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabaa04-cadf-4ac3-b34c-4a945b50e4e6 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:42.107497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a2452-cbae-4788-98cd-3a89c4ae89c4 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:42.142639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5685c-40ef-4ed4-9de9-84ed06f6d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.143561 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5685c-40ef-4ed4-9de9-84ed06f6d89a None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.143843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5685c-40ef-4ed4-9de9-84ed06f6d89a None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.144111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5685c-40ef-4ed4-9de9-84ed06f6d89a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.155732 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5685c-40ef-4ed4-9de9-84ed06f6d89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.157217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5685c-40ef-4ed4-9de9-84ed06f6d89a None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-1db5685c-40ef-4ed4-9de9-84ed06f6d89a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:42.172726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:42.173627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:42.174454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:42.174454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:42.174454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:42.174454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 172/348] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:42.174670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5685c-40ef-4ed4-9de9-84ed06f6d89a None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:42.181948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabaa04-cadf-4ac3-b34c-4a945b50e4e6 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:42.193622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:42.219280 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695704d-60b4-43ad-ad55-3cb631c2e2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.219886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695704d-60b4-43ad-ad55-3cb631c2e2cf None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.220163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695704d-60b4-43ad-ad55-3cb631c2e2cf None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.220470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695704d-60b4-43ad-ad55-3cb631c2e2cf None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.221120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695704d-60b4-43ad-ad55-3cb631c2e2cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.222967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695704d-60b4-43ad-ad55-3cb631c2e2cf None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.255059 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 177/349] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:42 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:42.255059 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695704d-60b4-43ad-ad55-3cb631c2e2cf None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:42.294447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5685c-40ef-4ed4-9de9-84ed06f6d89a None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:42.328612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:42.348217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03920-52c2-4cf8-bbfb-8e4c03afa3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.350057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03920-52c2-4cf8-bbfb-8e4c03afa3cb None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.350521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03920-52c2-4cf8-bbfb-8e4c03afa3cb None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.351072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03920-52c2-4cf8-bbfb-8e4c03afa3cb None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.352090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03920-52c2-4cf8-bbfb-8e4c03afa3cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:42.352572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03920-52c2-4cf8-bbfb-8e4c03afa3cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.354200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03920-52c2-4cf8-bbfb-8e4c03afa3cb None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.373149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 173/350] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:42.373149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03920-52c2-4cf8-bbfb-8e4c03afa3cb None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:42.391751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695704d-60b4-43ad-ad55-3cb631c2e2cf None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:42.436745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb262c-a32e-40f1-89d3-a54549a15600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.437327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb262c-a32e-40f1-89d3-a54549a15600 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.437608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb262c-a32e-40f1-89d3-a54549a15600 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.437881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb262c-a32e-40f1-89d3-a54549a15600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.452516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb262c-a32e-40f1-89d3-a54549a15600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.453993 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb262c-a32e-40f1-89d3-a54549a15600 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-1cdb262c-a32e-40f1-89d3-a54549a15600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:42.470406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:42.471373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:42.472209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:42.472209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:42.472209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:42.472209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 178/351] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:42.473327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb262c-a32e-40f1-89d3-a54549a15600 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:42.481998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:42.517916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03920-52c2-4cf8-bbfb-8e4c03afa3cb None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:42.558115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d2d1-f268-4454-834c-871bb85f3cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.558817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d2d1-f268-4454-834c-871bb85f3cec None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.559113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d2d1-f268-4454-834c-871bb85f3cec None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.559385 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d2d1-f268-4454-834c-871bb85f3cec None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.560038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7d2d1-f268-4454-834c-871bb85f3cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:42.560354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7d2d1-f268-4454-834c-871bb85f3cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.561754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7d2d1-f268-4454-834c-871bb85f3cec None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.578306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 174/352] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:42.578306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d2d1-f268-4454-834c-871bb85f3cec None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:42.594663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb262c-a32e-40f1-89d3-a54549a15600 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:42.602571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:42.631714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec159d0b-b282-4f98-a394-05788513b6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.632699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec159d0b-b282-4f98-a394-05788513b6a2 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.632978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec159d0b-b282-4f98-a394-05788513b6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.633247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec159d0b-b282-4f98-a394-05788513b6a2 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.633891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec159d0b-b282-4f98-a394-05788513b6a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.635750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec159d0b-b282-4f98-a394-05788513b6a2 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.658052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 179/353] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:42 2022] POST /identity/v3/users => generated 248 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:42.661285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec159d0b-b282-4f98-a394-05788513b6a2 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:42.690189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d2d1-f268-4454-834c-871bb85f3cec None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:42.726151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:42.732754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d85bb-f54d-48eb-95ea-37901661eac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.733969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d85bb-f54d-48eb-95ea-37901661eac6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.734476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d85bb-f54d-48eb-95ea-37901661eac6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.734962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d85bb-f54d-48eb-95ea-37901661eac6 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.735822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d85bb-f54d-48eb-95ea-37901661eac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:42.736334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d85bb-f54d-48eb-95ea-37901661eac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.738092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d85bb-f54d-48eb-95ea-37901661eac6 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.754221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 175/354] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:42.756313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dc963-e209-4c47-b29c-5ee422070236 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.756901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dc963-e209-4c47-b29c-5ee422070236 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.757432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dc963-e209-4c47-b29c-5ee422070236 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.760120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 176/355] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:42.761619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dc963-e209-4c47-b29c-5ee422070236 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:42.801671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec159d0b-b282-4f98-a394-05788513b6a2 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:42.849113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892bc2b-0f1f-4827-8141-7396f1ac161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.849910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892bc2b-0f1f-4827-8141-7396f1ac161f None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.850124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892bc2b-0f1f-4827-8141-7396f1ac161f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.850410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892bc2b-0f1f-4827-8141-7396f1ac161f None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.851139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892bc2b-0f1f-4827-8141-7396f1ac161f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.852535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892bc2b-0f1f-4827-8141-7396f1ac161f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.880268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 180/356] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:42 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:42.882060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cfb76-a2c2-41db-8d5c-f2d4861d122d None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.882570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cfb76-a2c2-41db-8d5c-f2d4861d122d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.883104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cfb76-a2c2-41db-8d5c-f2d4861d122d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.905286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0c3cfb76-a2c2-41db-8d5c-f2d4861d122d None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:42.906621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0c3cfb76-a2c2-41db-8d5c-f2d4861d122d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:42.907163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dc963-e209-4c47-b29c-5ee422070236 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:42.923239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:42.953869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a97a1-4779-4bed-9177-b0aed9c241e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:42.953869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a97a1-4779-4bed-9177-b0aed9c241e4 None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:42.954251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a97a1-4779-4bed-9177-b0aed9c241e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:42.954251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a97a1-4779-4bed-9177-b0aed9c241e4 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:42.955054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a97a1-4779-4bed-9177-b0aed9c241e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:42.956643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a97a1-4779-4bed-9177-b0aed9c241e4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:42.987440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 177/357] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:42 2022] POST /identity/v3/users => generated 249 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:43.007421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 181/358] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:43.301843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a97a1-4779-4bed-9177-b0aed9c241e4 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:43.354232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:43.417058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a97a1-4779-4bed-9177-b0aed9c241e4 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:43.455424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:43.456045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:43.456328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:43.456636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:43.457319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:43.458758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:43.469385 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.sql.core [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 20 14:28:43.469385 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 20 14:28:43.469385 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [parameters: {'id': 'a10061d3deee40f1ba4efc3535f2a554', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 20 14:28:43.469385 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82085) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result.read() Oct 20 14:28:43.489333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return ref.to_dict() Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:28:43.490248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application yield resource Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise self.value Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application session.commit() Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 14:28:43.491169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.session.flush() Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application compat.raise_( Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 14:28:43.491995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:28:43.494440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result.read() Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:28:43.497116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [parameters: {'id': 'a10061d3deee40f1ba4efc3535f2a554', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:43.499075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 20 14:28:43.501168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:43.506264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 178/359] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Oct 20 14:28:43 2022] POST /identity/v3/roles => generated 144 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:43.515441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cfb76-a2c2-41db-8d5c-f2d4861d122d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:43.639729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cfb76-a2c2-41db-8d5c-f2d4861d122d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:43.677377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d67de5-7401-4cf7-85e2-d4707457ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:43.678379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d67de5-7401-4cf7-85e2-d4707457ad72 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:43.678860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d67de5-7401-4cf7-85e2-d4707457ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:43.679352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d67de5-7401-4cf7-85e2-d4707457ad72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:43.690180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d67de5-7401-4cf7-85e2-d4707457ad72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-e7d67de5-7401-4cf7-85e2-d4707457ad72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:43.695449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:43.696310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 14:28:43.697466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:43.700131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 182/360] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:28:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:43.706134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:43.865711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ebe30-c2d5-4a0c-816c-4f8d63db8c4c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:43.885108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:43.922394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dafc6-6c14-4762-835e-cb72c3cea6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:43.922925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dafc6-6c14-4762-835e-cb72c3cea6a9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:43.923204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dafc6-6c14-4762-835e-cb72c3cea6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:43.923476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dafc6-6c14-4762-835e-cb72c3cea6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:43.924148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dafc6-6c14-4762-835e-cb72c3cea6a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:43.924493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dafc6-6c14-4762-835e-cb72c3cea6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:43.925873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dafc6-6c14-4762-835e-cb72c3cea6a9 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:43.934673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 179/361] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:45.015989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86edf4-91d8-4924-b3be-ac8cc3e512c3 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.016551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86edf4-91d8-4924-b3be-ac8cc3e512c3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.016551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86edf4-91d8-4924-b3be-ac8cc3e512c3 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.018144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 183/362] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:45.023492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e702b7-0c48-4c4d-b797-618f5510fe35 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.023880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e702b7-0c48-4c4d-b797-618f5510fe35 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.024056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e702b7-0c48-4c4d-b797-618f5510fe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.053506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-c3e702b7-0c48-4c4d-b797-618f5510fe35 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:45.054741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c3e702b7-0c48-4c4d-b797-618f5510fe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:45.138945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 180/363] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:45.470971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a41d2b-7ff7-4339-96ab-718ef0481fbd None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.471337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a41d2b-7ff7-4339-96ab-718ef0481fbd None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.471537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a41d2b-7ff7-4339-96ab-718ef0481fbd None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.474423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 184/364] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:45.479450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdd4e6-cea0-45ed-a452-f482ee5d2267 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.480004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdd4e6-cea0-45ed-a452-f482ee5d2267 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.480526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdd4e6-cea0-45ed-a452-f482ee5d2267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.501228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-f5cdd4e6-cea0-45ed-a452-f482ee5d2267 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:45.502440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f5cdd4e6-cea0-45ed-a452-f482ee5d2267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:45.527965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a41d2b-7ff7-4339-96ab-718ef0481fbd None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:45.543225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:45.626904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 181/365] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:45.652495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891432d-8a40-4574-9d52-1038176b4acf None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.652903 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891432d-8a40-4574-9d52-1038176b4acf None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.653199 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891432d-8a40-4574-9d52-1038176b4acf None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.654808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 182/366] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:28:45.681795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e65b2-9670-4467-b951-86ff1e4f365f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.682117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e65b2-9670-4467-b951-86ff1e4f365f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.682388 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e65b2-9670-4467-b951-86ff1e4f365f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.704431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a41d2b-7ff7-4339-96ab-718ef0481fbd None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:45.705381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-046e65b2-9670-4467-b951-86ff1e4f365f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:45.706943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-046e65b2-9670-4467-b951-86ff1e4f365f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:45.737937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:45.739917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87546c-6ca5-4068-9a1b-502a61c3a973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:45.740611 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87546c-6ca5-4068-9a1b-502a61c3a973 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.740897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87546c-6ca5-4068-9a1b-502a61c3a973 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.741190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87546c-6ca5-4068-9a1b-502a61c3a973 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.753347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87546c-6ca5-4068-9a1b-502a61c3a973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-2b87546c-6ca5-4068-9a1b-502a61c3a973 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:45.758422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:45.759285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:45.760358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 185/367] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:45.761319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63825538-bc41-4c81-a606-674bdcc82cba None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.761557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63825538-bc41-4c81-a606-674bdcc82cba None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.761872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63825538-bc41-4c81-a606-674bdcc82cba None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.763351 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 186/368] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:45.767942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63825538-bc41-4c81-a606-674bdcc82cba None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:45.801580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 183/369] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:45.801785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51073b2-be00-417c-b3a1-e54d379628bf None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.802028 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51073b2-be00-417c-b3a1-e54d379628bf None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.803411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51073b2-be00-417c-b3a1-e54d379628bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.838641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-a51073b2-be00-417c-b3a1-e54d379628bf None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:45.840408 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a51073b2-be00-417c-b3a1-e54d379628bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:45.872916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:45.933833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63825538-bc41-4c81-a606-674bdcc82cba None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:45.952391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 184/370] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:45.953415 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51073b2-be00-417c-b3a1-e54d379628bf None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:45.974307 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78b091-e25c-4d5b-9652-833a813b7147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:45.975000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78b091-e25c-4d5b-9652-833a813b7147 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:45.975380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78b091-e25c-4d5b-9652-833a813b7147 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:45.975663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78b091-e25c-4d5b-9652-833a813b7147 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:45.976494 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78b091-e25c-4d5b-9652-833a813b7147 None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:45.977577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78b091-e25c-4d5b-9652-833a813b7147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:45.979279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78b091-e25c-4d5b-9652-833a813b7147 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:45.989562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 187/371] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:45.995261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78b091-e25c-4d5b-9652-833a813b7147 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.058784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:46.067941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51073b2-be00-417c-b3a1-e54d379628bf None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.123831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0959f9-9bc3-4674-9326-9edcb9ccc595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.124501 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0959f9-9bc3-4674-9326-9edcb9ccc595 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.124781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0959f9-9bc3-4674-9326-9edcb9ccc595 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.125060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0959f9-9bc3-4674-9326-9edcb9ccc595 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.144798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0959f9-9bc3-4674-9326-9edcb9ccc595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-ae0959f9-9bc3-4674-9326-9edcb9ccc595 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:46.149687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:46.150546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:46.151323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 185/372] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:46.158859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0959f9-9bc3-4674-9326-9edcb9ccc595 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.185242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78b091-e25c-4d5b-9652-833a813b7147 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.218483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:46.225792 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ade86c-337a-4dff-a88e-a43687257334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.226428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ade86c-337a-4dff-a88e-a43687257334 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.226711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ade86c-337a-4dff-a88e-a43687257334 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.226981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ade86c-337a-4dff-a88e-a43687257334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.238123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ade86c-337a-4dff-a88e-a43687257334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:46.239702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ade86c-337a-4dff-a88e-a43687257334 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-88ade86c-337a-4dff-a88e-a43687257334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:46.255784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:46.256721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:46.257917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:46.257917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:46.257917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:46.257917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 188/373] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:46.258208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ade86c-337a-4dff-a88e-a43687257334 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.295346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0959f9-9bc3-4674-9326-9edcb9ccc595 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.331411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db23f6b-24da-4c45-9c46-03f0f935737f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.332024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db23f6b-24da-4c45-9c46-03f0f935737f None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.332306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db23f6b-24da-4c45-9c46-03f0f935737f None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.332609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db23f6b-24da-4c45-9c46-03f0f935737f None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.333276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db23f6b-24da-4c45-9c46-03f0f935737f None admin] RBAC: Adding query filter params (name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:46.333579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db23f6b-24da-4c45-9c46-03f0f935737f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:46.334956 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db23f6b-24da-4c45-9c46-03f0f935737f None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:46.352823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 186/374] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:46.354001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db23f6b-24da-4c45-9c46-03f0f935737f None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.373148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ade86c-337a-4dff-a88e-a43687257334 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.374076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:46.412646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862a1e7-6164-42ca-81db-d4a9efd5a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.413679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862a1e7-6164-42ca-81db-d4a9efd5a326 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.414411 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862a1e7-6164-42ca-81db-d4a9efd5a326 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.415128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862a1e7-6164-42ca-81db-d4a9efd5a326 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.427620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862a1e7-6164-42ca-81db-d4a9efd5a326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-2862a1e7-6164-42ca-81db-d4a9efd5a326 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:46.434486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:46.435601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:46.436625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:46.439445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 189/375] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:46.440682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862a1e7-6164-42ca-81db-d4a9efd5a326 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.465328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db23f6b-24da-4c45-9c46-03f0f935737f None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.493207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:46.501324 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811b8f0-2f21-4596-abe7-4e7d9bde9f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.501944 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811b8f0-2f21-4596-abe7-4e7d9bde9f09 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.502252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811b8f0-2f21-4596-abe7-4e7d9bde9f09 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.502502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811b8f0-2f21-4596-abe7-4e7d9bde9f09 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.503148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811b8f0-2f21-4596-abe7-4e7d9bde9f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:46.503452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811b8f0-2f21-4596-abe7-4e7d9bde9f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:46.504867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811b8f0-2f21-4596-abe7-4e7d9bde9f09 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:46.521277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 187/376] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:46.521277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811b8f0-2f21-4596-abe7-4e7d9bde9f09 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.557430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862a1e7-6164-42ca-81db-d4a9efd5a326 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.597011 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4c7a6-30c9-4e06-b540-9636982ae561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.597649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4c7a6-30c9-4e06-b540-9636982ae561 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.597932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4c7a6-30c9-4e06-b540-9636982ae561 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.598208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4c7a6-30c9-4e06-b540-9636982ae561 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.603147 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:46.610620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4c7a6-30c9-4e06-b540-9636982ae561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-9ca4c7a6-30c9-4e06-b540-9636982ae561 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:46.616016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:46.616932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:46.617987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 190/377] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:46.618641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4c7a6-30c9-4e06-b540-9636982ae561 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.632640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811b8f0-2f21-4596-abe7-4e7d9bde9f09 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.667896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ca461-fdae-413f-b0b8-a81599ca4b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.668623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ca461-fdae-413f-b0b8-a81599ca4b42 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.668903 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ca461-fdae-413f-b0b8-a81599ca4b42 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.669175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ca461-fdae-413f-b0b8-a81599ca4b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.679255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ca461-fdae-413f-b0b8-a81599ca4b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:46.680767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ca461-fdae-413f-b0b8-a81599ca4b42 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-5e9ca461-fdae-413f-b0b8-a81599ca4b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:46.695967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:46.696897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:46.697720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:46.697720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:46.697720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:46.697720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 188/378] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:46.697944 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ca461-fdae-413f-b0b8-a81599ca4b42 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.728896 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4c7a6-30c9-4e06-b540-9636982ae561 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.730389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:46.763606 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a1ad8-f617-4db5-81ae-f5f8f6fed605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.764212 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a1ad8-f617-4db5-81ae-f5f8f6fed605 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.764527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a1ad8-f617-4db5-81ae-f5f8f6fed605 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.764804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a1ad8-f617-4db5-81ae-f5f8f6fed605 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.765468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a1ad8-f617-4db5-81ae-f5f8f6fed605 None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:46.765777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a1ad8-f617-4db5-81ae-f5f8f6fed605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:46.767135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a1ad8-f617-4db5-81ae-f5f8f6fed605 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:46.774857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 191/379] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:46.775632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a1ad8-f617-4db5-81ae-f5f8f6fed605 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.800908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ca461-fdae-413f-b0b8-a81599ca4b42 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.826354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:46.834268 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf25a0a-cf6a-405b-8014-8b3736be1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.834836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf25a0a-cf6a-405b-8014-8b3736be1de9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.835121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf25a0a-cf6a-405b-8014-8b3736be1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.835402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf25a0a-cf6a-405b-8014-8b3736be1de9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.842768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf25a0a-cf6a-405b-8014-8b3736be1de9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:46.844221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf25a0a-cf6a-405b-8014-8b3736be1de9 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-adf25a0a-cf6a-405b-8014-8b3736be1de9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:46.857286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:46.858348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 189/380] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:46.859508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ae9b-dbeb-4263-a2f6-066591014878 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.860270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ae9b-dbeb-4263-a2f6-066591014878 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.860270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ae9b-dbeb-4263-a2f6-066591014878 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.861411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 190/381] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:46.861947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ae9b-dbeb-4263-a2f6-066591014878 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.875943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a1ad8-f617-4db5-81ae-f5f8f6fed605 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:46.908568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f7e20-8f88-4ce3-b76d-8ba3639f4b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:46.909170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f7e20-8f88-4ce3-b76d-8ba3639f4b59 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:46.909447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f7e20-8f88-4ce3-b76d-8ba3639f4b59 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:46.909718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f7e20-8f88-4ce3-b76d-8ba3639f4b59 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:46.910364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f7e20-8f88-4ce3-b76d-8ba3639f4b59 None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:46.910661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f7e20-8f88-4ce3-b76d-8ba3639f4b59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:46.912012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f7e20-8f88-4ce3-b76d-8ba3639f4b59 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:46.920080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 192/382] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:46.920883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f7e20-8f88-4ce3-b76d-8ba3639f4b59 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:46.947023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:46.967686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ae9b-dbeb-4263-a2f6-066591014878 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.004473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fa60a-523a-4bf0-8c4a-9e2f25bd01fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.005074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fa60a-523a-4bf0-8c4a-9e2f25bd01fb None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.005354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fa60a-523a-4bf0-8c4a-9e2f25bd01fb None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.005628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fa60a-523a-4bf0-8c4a-9e2f25bd01fb None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.006266 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fa60a-523a-4bf0-8c4a-9e2f25bd01fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:47.006574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fa60a-523a-4bf0-8c4a-9e2f25bd01fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.007958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fa60a-523a-4bf0-8c4a-9e2f25bd01fb None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.019506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 191/383] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:47.020205 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fa60a-523a-4bf0-8c4a-9e2f25bd01fb None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.024152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f7e20-8f88-4ce3-b76d-8ba3639f4b59 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.056797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1030ed-c47d-46e3-93e4-76215caa57eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.057429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1030ed-c47d-46e3-93e4-76215caa57eb None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.058203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1030ed-c47d-46e3-93e4-76215caa57eb None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.058478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1030ed-c47d-46e3-93e4-76215caa57eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.067375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:47.068955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1030ed-c47d-46e3-93e4-76215caa57eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.070458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1030ed-c47d-46e3-93e4-76215caa57eb None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-4f1030ed-c47d-46e3-93e4-76215caa57eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:47.085249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:47.086185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:47.087003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:47.087003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:47.087003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:47.087003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 193/384] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:47.087510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f2e44-ee22-4061-8ef7-99a0fedc11da None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.087790 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f2e44-ee22-4061-8ef7-99a0fedc11da None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.087994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f2e44-ee22-4061-8ef7-99a0fedc11da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.105642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f80f2e44-ee22-4061-8ef7-99a0fedc11da None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:47.106831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f80f2e44-ee22-4061-8ef7-99a0fedc11da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:47.123140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fa60a-523a-4bf0-8c4a-9e2f25bd01fb None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.156946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0773639-e9f0-4788-811e-eab53dc155fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.157547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0773639-e9f0-4788-811e-eab53dc155fd None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.157829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0773639-e9f0-4788-811e-eab53dc155fd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.158107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0773639-e9f0-4788-811e-eab53dc155fd None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.158841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0773639-e9f0-4788-811e-eab53dc155fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:47.159147 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0773639-e9f0-4788-811e-eab53dc155fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.160613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0773639-e9f0-4788-811e-eab53dc155fd None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.171681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 194/385] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:47.172544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f2e44-ee22-4061-8ef7-99a0fedc11da None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.182115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:47.197835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 192/386] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:47.199096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0773639-e9f0-4788-811e-eab53dc155fd None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.295125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f2e44-ee22-4061-8ef7-99a0fedc11da None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.299415 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0773639-e9f0-4788-811e-eab53dc155fd None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.308889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:47.333098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f914ac-3694-45bc-9bba-a40d59a38910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.334101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f914ac-3694-45bc-9bba-a40d59a38910 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.334542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f914ac-3694-45bc-9bba-a40d59a38910 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.334777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ba003-65eb-42d8-b6f2-6e3dbd4154b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.335120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f914ac-3694-45bc-9bba-a40d59a38910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.335394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ba003-65eb-42d8-b6f2-6e3dbd4154b0 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.335673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ba003-65eb-42d8-b6f2-6e3dbd4154b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.335947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ba003-65eb-42d8-b6f2-6e3dbd4154b0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.345743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ba003-65eb-42d8-b6f2-6e3dbd4154b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.347264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ba003-65eb-42d8-b6f2-6e3dbd4154b0 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.352169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f914ac-3694-45bc-9bba-a40d59a38910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.353935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f914ac-3694-45bc-9bba-a40d59a38910 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-bb4ba003-65eb-42d8-b6f2-6e3dbd4154b0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:47.360480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:47.361313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:47.362172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 193/387] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:47.362807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ba003-65eb-42d8-b6f2-6e3dbd4154b0 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-b8f914ac-3694-45bc-9bba-a40d59a38910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:47.374569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:47.375655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:47.376700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:47.376700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:47.376700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:47.378622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 195/388] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:47.379694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f914ac-3694-45bc-9bba-a40d59a38910 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.444282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:47.467668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ba003-65eb-42d8-b6f2-6e3dbd4154b0 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.485775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f914ac-3694-45bc-9bba-a40d59a38910 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.501438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de7c75-b338-46c7-b9a7-11a0f6d5fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.502034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de7c75-b338-46c7-b9a7-11a0f6d5fa8b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.502310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de7c75-b338-46c7-b9a7-11a0f6d5fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.502580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de7c75-b338-46c7-b9a7-11a0f6d5fa8b None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.503211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de7c75-b338-46c7-b9a7-11a0f6d5fa8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:47.503510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de7c75-b338-46c7-b9a7-11a0f6d5fa8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.504902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de7c75-b338-46c7-b9a7-11a0f6d5fa8b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.516327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 194/389] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:47.517124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de7c75-b338-46c7-b9a7-11a0f6d5fa8b None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.525463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb09461-a484-496a-b1cf-6f24c35f17b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.526167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09461-a484-496a-b1cf-6f24c35f17b6 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.526528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09461-a484-496a-b1cf-6f24c35f17b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.526791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09461-a484-496a-b1cf-6f24c35f17b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.537375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb09461-a484-496a-b1cf-6f24c35f17b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.538813 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb09461-a484-496a-b1cf-6f24c35f17b6 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.543608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-1eb09461-a484-496a-b1cf-6f24c35f17b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:47.554424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:47.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:47.556447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:47.556447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:47.556447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:47.556447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 196/390] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:47.556447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb09461-a484-496a-b1cf-6f24c35f17b6 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.614692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de7c75-b338-46c7-b9a7-11a0f6d5fa8b None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.647873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94614f-d24b-44d2-8c6d-2ecfeab82540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.648509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94614f-d24b-44d2-8c6d-2ecfeab82540 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.648792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94614f-d24b-44d2-8c6d-2ecfeab82540 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.649069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94614f-d24b-44d2-8c6d-2ecfeab82540 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.649790 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94614f-d24b-44d2-8c6d-2ecfeab82540 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:47.650097 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94614f-d24b-44d2-8c6d-2ecfeab82540 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.651628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94614f-d24b-44d2-8c6d-2ecfeab82540 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.656934 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb09461-a484-496a-b1cf-6f24c35f17b6 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.668694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 195/391] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:47.669494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94614f-d24b-44d2-8c6d-2ecfeab82540 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.670031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:47.692446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ffdb5-c8c1-4119-b418-398fe5007ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.693041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ffdb5-c8c1-4119-b418-398fe5007ce2 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.693316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ffdb5-c8c1-4119-b418-398fe5007ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.693581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ffdb5-c8c1-4119-b418-398fe5007ce2 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.694213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ffdb5-c8c1-4119-b418-398fe5007ce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:47.694512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ffdb5-c8c1-4119-b418-398fe5007ce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.695872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ffdb5-c8c1-4119-b418-398fe5007ce2 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.707870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 197/392] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:47.708672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ffdb5-c8c1-4119-b418-398fe5007ce2 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.755211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:47.771127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94614f-d24b-44d2-8c6d-2ecfeab82540 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.805597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1a602-110a-4731-810c-ca55b59eb432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.806227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1a602-110a-4731-810c-ca55b59eb432 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.806512 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1a602-110a-4731-810c-ca55b59eb432 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.806789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1a602-110a-4731-810c-ca55b59eb432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.809712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ffdb5-c8c1-4119-b418-398fe5007ce2 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.817202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1a602-110a-4731-810c-ca55b59eb432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.818672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1a602-110a-4731-810c-ca55b59eb432 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-6db1a602-110a-4731-810c-ca55b59eb432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:47.832957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:47.833821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:47.834667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:47.834667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:47.834667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:47.834667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 196/393] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:47.834882 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1a602-110a-4731-810c-ca55b59eb432 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.844625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f839c7d-1283-45b4-be63-78d34a50047e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.845250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f839c7d-1283-45b4-be63-78d34a50047e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.845533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f839c7d-1283-45b4-be63-78d34a50047e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.845834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f839c7d-1283-45b4-be63-78d34a50047e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.852788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f839c7d-1283-45b4-be63-78d34a50047e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.854235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f839c7d-1283-45b4-be63-78d34a50047e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-4f839c7d-1283-45b4-be63-78d34a50047e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:47.866707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:47.867801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:47.868688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 198/394] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:47.869226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f839c7d-1283-45b4-be63-78d34a50047e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:47.900518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:47.945640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1a602-110a-4731-810c-ca55b59eb432 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.968876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f839c7d-1283-45b4-be63-78d34a50047e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:47.978696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6bddb-1453-4749-ac8a-e199e7b40a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:47.979293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6bddb-1453-4749-ac8a-e199e7b40a98 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:47.979652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6bddb-1453-4749-ac8a-e199e7b40a98 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:47.980505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6bddb-1453-4749-ac8a-e199e7b40a98 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:47.981145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6bddb-1453-4749-ac8a-e199e7b40a98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:47.981449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6bddb-1453-4749-ac8a-e199e7b40a98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:47.982831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6bddb-1453-4749-ac8a-e199e7b40a98 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:47.985045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:47.994744 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 197/395] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:47.995573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6bddb-1453-4749-ac8a-e199e7b40a98 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.007012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea71198-4d14-4839-b4ed-925f9b8a22e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.007572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea71198-4d14-4839-b4ed-925f9b8a22e8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.007844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea71198-4d14-4839-b4ed-925f9b8a22e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.008116 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea71198-4d14-4839-b4ed-925f9b8a22e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.018477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea71198-4d14-4839-b4ed-925f9b8a22e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.019920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea71198-4d14-4839-b4ed-925f9b8a22e8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-0ea71198-4d14-4839-b4ed-925f9b8a22e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:48.034149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:48.035042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:48.035865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:48.035865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:48.035865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:48.035865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 199/396] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:48.036080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea71198-4d14-4839-b4ed-925f9b8a22e8 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.096959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6bddb-1453-4749-ac8a-e199e7b40a98 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.117236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:48.129487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b27f6-c5b8-4c5a-81ba-6e76b6b43f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.130115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b27f6-c5b8-4c5a-81ba-6e76b6b43f00 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.130399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b27f6-c5b8-4c5a-81ba-6e76b6b43f00 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.130680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b27f6-c5b8-4c5a-81ba-6e76b6b43f00 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.133320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea71198-4d14-4839-b4ed-925f9b8a22e8 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.138396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b27f6-c5b8-4c5a-81ba-6e76b6b43f00 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.139835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b27f6-c5b8-4c5a-81ba-6e76b6b43f00 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-a80b27f6-c5b8-4c5a-81ba-6e76b6b43f00 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:48.150812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:48.151793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 198/397] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:48.152919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b27f6-c5b8-4c5a-81ba-6e76b6b43f00 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.168555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9887540-085d-4a30-9bff-7962db9a8f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.169224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887540-085d-4a30-9bff-7962db9a8f2c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.169501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887540-085d-4a30-9bff-7962db9a8f2c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.169766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887540-085d-4a30-9bff-7962db9a8f2c None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.170392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9887540-085d-4a30-9bff-7962db9a8f2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:48.170694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9887540-085d-4a30-9bff-7962db9a8f2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.172048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9887540-085d-4a30-9bff-7962db9a8f2c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.184232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 200/398] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:48.185073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9887540-085d-4a30-9bff-7962db9a8f2c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.213219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:48.256625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b27f6-c5b8-4c5a-81ba-6e76b6b43f00 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.281848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9887540-085d-4a30-9bff-7962db9a8f2c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.289624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabccc3-5068-40d5-ad3a-fe8e2a8a0c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.290221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabccc3-5068-40d5-ad3a-fe8e2a8a0c04 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.290496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabccc3-5068-40d5-ad3a-fe8e2a8a0c04 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.290763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabccc3-5068-40d5-ad3a-fe8e2a8a0c04 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.291472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabccc3-5068-40d5-ad3a-fe8e2a8a0c04 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:48.291775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabccc3-5068-40d5-ad3a-fe8e2a8a0c04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.294037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabccc3-5068-40d5-ad3a-fe8e2a8a0c04 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.310532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 199/399] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:48.311312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabccc3-5068-40d5-ad3a-fe8e2a8a0c04 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.318149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a171b-96b1-4075-81bc-2ff513469d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.318773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a171b-96b1-4075-81bc-2ff513469d19 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.319048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a171b-96b1-4075-81bc-2ff513469d19 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.319320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a171b-96b1-4075-81bc-2ff513469d19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.326551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a171b-96b1-4075-81bc-2ff513469d19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.327987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a171b-96b1-4075-81bc-2ff513469d19 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-438a171b-96b1-4075-81bc-2ff513469d19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:28:48.340588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:48.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:48.342294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:28:48.342294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:28:48.342294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:28:48.342294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:28:48.342294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:28:48.342294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:28:48.342294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:48.342294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:48.342294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 201/400] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:48.342743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a171b-96b1-4075-81bc-2ff513469d19 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.346132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:48.415369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabccc3-5068-40d5-ad3a-fe8e2a8a0c04 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.429384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:48.443895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a171b-96b1-4075-81bc-2ff513469d19 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.450094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86a01e-c049-434b-a6c7-c73f2a5af5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.450701 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86a01e-c049-434b-a6c7-c73f2a5af5da None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.450983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86a01e-c049-434b-a6c7-c73f2a5af5da None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.451256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86a01e-c049-434b-a6c7-c73f2a5af5da None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.451886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86a01e-c049-434b-a6c7-c73f2a5af5da None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:48.452187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86a01e-c049-434b-a6c7-c73f2a5af5da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.453591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86a01e-c049-434b-a6c7-c73f2a5af5da None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.465125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 200/401] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:48.465887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86a01e-c049-434b-a6c7-c73f2a5af5da None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.483007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f74d9-a499-405a-b5aa-25093f57f4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.483614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f74d9-a499-405a-b5aa-25093f57f4ca None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.483895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f74d9-a499-405a-b5aa-25093f57f4ca None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.484166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f74d9-a499-405a-b5aa-25093f57f4ca None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.484918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f74d9-a499-405a-b5aa-25093f57f4ca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:48.485225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f74d9-a499-405a-b5aa-25093f57f4ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.486666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f74d9-a499-405a-b5aa-25093f57f4ca None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.529385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 202/402] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:48.529385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f74d9-a499-405a-b5aa-25093f57f4ca None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.563255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86a01e-c049-434b-a6c7-c73f2a5af5da None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.585392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:48.597603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72db78-2601-4a63-83d3-80ca8003b3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.598277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72db78-2601-4a63-83d3-80ca8003b3bd None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.598621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72db78-2601-4a63-83d3-80ca8003b3bd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.598960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72db78-2601-4a63-83d3-80ca8003b3bd None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.599660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72db78-2601-4a63-83d3-80ca8003b3bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.601207 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72db78-2601-4a63-83d3-80ca8003b3bd None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.624461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 201/403] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:28:48 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:28:48.625243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72db78-2601-4a63-83d3-80ca8003b3bd None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.625605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f74d9-a499-405a-b5aa-25093f57f4ca None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.664768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4d4ff-3f3e-4dbe-9136-92fd35c112a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.665423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4d4ff-3f3e-4dbe-9136-92fd35c112a0 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.665950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4d4ff-3f3e-4dbe-9136-92fd35c112a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.666193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4d4ff-3f3e-4dbe-9136-92fd35c112a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.674436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:48.678609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4d4ff-3f3e-4dbe-9136-92fd35c112a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.680382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4d4ff-3f3e-4dbe-9136-92fd35c112a0 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-f9e4d4ff-3f3e-4dbe-9136-92fd35c112a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:48.697711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:48.698900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:48.699824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:48.699824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:48.699824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:48.699824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 203/404] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:48.699824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4d4ff-3f3e-4dbe-9136-92fd35c112a0 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.760432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72db78-2601-4a63-83d3-80ca8003b3bd None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.798959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978df5b-8b34-4925-8eee-12ccbc0620fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.799579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978df5b-8b34-4925-8eee-12ccbc0620fc None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.799860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978df5b-8b34-4925-8eee-12ccbc0620fc None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.800131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978df5b-8b34-4925-8eee-12ccbc0620fc None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.800852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978df5b-8b34-4925-8eee-12ccbc0620fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:48.801161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978df5b-8b34-4925-8eee-12ccbc0620fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.802614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978df5b-8b34-4925-8eee-12ccbc0620fc None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.822128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4d4ff-3f3e-4dbe-9136-92fd35c112a0 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.825284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 202/405] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:48.826115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978df5b-8b34-4925-8eee-12ccbc0620fc None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.843407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:48.867028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987600b9-142d-4080-9aa0-9212a4dd9900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.867914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987600b9-142d-4080-9aa0-9212a4dd9900 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.868195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987600b9-142d-4080-9aa0-9212a4dd9900 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.868610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987600b9-142d-4080-9aa0-9212a4dd9900 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.876017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987600b9-142d-4080-9aa0-9212a4dd9900 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.877619 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987600b9-142d-4080-9aa0-9212a4dd9900 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-987600b9-142d-4080-9aa0-9212a4dd9900 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:28:48.890511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:48.891365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:48.892182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:28:48.892182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:28:48.892182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:28:48.892182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:28:48.892182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:28:48.892182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:28:48.892182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:48.892182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:48.892182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 204/406] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:48.892677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987600b9-142d-4080-9aa0-9212a4dd9900 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:48.922153 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:48.932626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978df5b-8b34-4925-8eee-12ccbc0620fc None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:48.966657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7f60b-949c-40ab-8a97-e348bbee5355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:48.967287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7f60b-949c-40ab-8a97-e348bbee5355 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:48.967562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7f60b-949c-40ab-8a97-e348bbee5355 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:48.967831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7f60b-949c-40ab-8a97-e348bbee5355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:48.977957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7f60b-949c-40ab-8a97-e348bbee5355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:48.979406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7f60b-949c-40ab-8a97-e348bbee5355 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-0db7f60b-949c-40ab-8a97-e348bbee5355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:48.995020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:48.996778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:48.997621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:48.997621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:48.997621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:48.997621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 203/407] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:48.997621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7f60b-949c-40ab-8a97-e348bbee5355 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:49.009631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987600b9-142d-4080-9aa0-9212a4dd9900 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:49.045205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147288a-6a8b-4cc6-8f3b-7a1ea7ca3dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:49.045803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147288a-6a8b-4cc6-8f3b-7a1ea7ca3dbb None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:49.046079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147288a-6a8b-4cc6-8f3b-7a1ea7ca3dbb None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:49.046524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147288a-6a8b-4cc6-8f3b-7a1ea7ca3dbb None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:49.047551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147288a-6a8b-4cc6-8f3b-7a1ea7ca3dbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:49.047866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147288a-6a8b-4cc6-8f3b-7a1ea7ca3dbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:49.049276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147288a-6a8b-4cc6-8f3b-7a1ea7ca3dbb None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:49.062035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 205/408] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:49.062673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147288a-6a8b-4cc6-8f3b-7a1ea7ca3dbb None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:49.097444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:49.108600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7f60b-949c-40ab-8a97-e348bbee5355 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:49.144714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c08fb-573d-46cb-b733-58c3fe4df3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:49.145614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c08fb-573d-46cb-b733-58c3fe4df3df None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:49.145900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c08fb-573d-46cb-b733-58c3fe4df3df None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:49.146191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c08fb-573d-46cb-b733-58c3fe4df3df None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/9647f8caa6cd4208800e2371fdf28301/roles/770887c5c808476d8505bf8b8fc5f27a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:49.158564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:49.161881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c08fb-573d-46cb-b733-58c3fe4df3df None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=9647f8caa6cd4208800e2371fdf28301, role_id=770887c5c808476d8505bf8b8fc5f27a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:49.164076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c08fb-573d-46cb-b733-58c3fe4df3df None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:49.172433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147288a-6a8b-4cc6-8f3b-7a1ea7ca3dbb None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:49.177799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 204/409] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:48 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/9647f8caa6cd4208800e2371fdf28301/roles/770887c5c808476d8505bf8b8fc5f27a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:49.178673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c08fb-573d-46cb-b733-58c3fe4df3df None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:49.220662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabfeb7-112b-44d3-9e25-1638b44627b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:49.221526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabfeb7-112b-44d3-9e25-1638b44627b0 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:49.221942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabfeb7-112b-44d3-9e25-1638b44627b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:49.222350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabfeb7-112b-44d3-9e25-1638b44627b0 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:49.223211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabfeb7-112b-44d3-9e25-1638b44627b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:49.223653 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabfeb7-112b-44d3-9e25-1638b44627b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:49.226175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabfeb7-112b-44d3-9e25-1638b44627b0 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:49.249367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 206/410] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:49.250484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabfeb7-112b-44d3-9e25-1638b44627b0 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:49.316209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c08fb-573d-46cb-b733-58c3fe4df3df None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:49.374251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ea6a9-f10f-4c74-bbe8-7de3b5d94a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:49.375409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ea6a9-f10f-4c74-bbe8-7de3b5d94a70 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:49.375829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ea6a9-f10f-4c74-bbe8-7de3b5d94a70 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:49.376444 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ea6a9-f10f-4c74-bbe8-7de3b5d94a70 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:49.377621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ea6a9-f10f-4c74-bbe8-7de3b5d94a70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:49.378075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ea6a9-f10f-4c74-bbe8-7de3b5d94a70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:49.379948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ea6a9-f10f-4c74-bbe8-7de3b5d94a70 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:49.380076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:49.389934 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabfeb7-112b-44d3-9e25-1638b44627b0 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:49.396727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 205/411] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:49.402359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ea6a9-f10f-4c74-bbe8-7de3b5d94a70 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:49.431461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1a4bf-2ca5-4b50-bf57-b7c50dad2b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:49.432279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1a4bf-2ca5-4b50-bf57-b7c50dad2b3b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:49.432468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1a4bf-2ca5-4b50-bf57-b7c50dad2b3b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:49.433255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1a4bf-2ca5-4b50-bf57-b7c50dad2b3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:49.442854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1a4bf-2ca5-4b50-bf57-b7c50dad2b3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:49.444310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1a4bf-2ca5-4b50-bf57-b7c50dad2b3b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-09f1a4bf-2ca5-4b50-bf57-b7c50dad2b3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:28:49.456385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:49.457245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:49.458056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:28:49.458056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:28:49.458056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:28:49.458056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:28:49.458056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:28:49.458056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:28:49.458056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:49.458056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:49.458056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 207/412] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:49.458508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1a4bf-2ca5-4b50-bf57-b7c50dad2b3b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:49.468396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:49.522730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ea6a9-f10f-4c74-bbe8-7de3b5d94a70 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:49.574481 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d0cf8-1fb1-477a-8427-acf30188c3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:49.575305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d0cf8-1fb1-477a-8427-acf30188c3a7 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:49.575657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d0cf8-1fb1-477a-8427-acf30188c3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:49.576001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d0cf8-1fb1-477a-8427-acf30188c3a7 None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/cacfce5c9d07471cbe1713a4d6aa29f0/roles/770887c5c808476d8505bf8b8fc5f27a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:49.592788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1a4bf-2ca5-4b50-bf57-b7c50dad2b3b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:49.597726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d0cf8-1fb1-477a-8427-acf30188c3a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=cacfce5c9d07471cbe1713a4d6aa29f0, role_id=770887c5c808476d8505bf8b8fc5f27a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:49.599269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d0cf8-1fb1-477a-8427-acf30188c3a7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:49.616588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 206/413] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:49 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/cacfce5c9d07471cbe1713a4d6aa29f0/roles/770887c5c808476d8505bf8b8fc5f27a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:49.621419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d0cf8-1fb1-477a-8427-acf30188c3a7 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:49.650329 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fc59a-91da-4324-a3e6-4e35f486040b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:49.652507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fc59a-91da-4324-a3e6-4e35f486040b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:49.654340 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fc59a-91da-4324-a3e6-4e35f486040b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:49.655454 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fc59a-91da-4324-a3e6-4e35f486040b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:49.674748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fc59a-91da-4324-a3e6-4e35f486040b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:49.676605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fc59a-91da-4324-a3e6-4e35f486040b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-796fc59a-91da-4324-a3e6-4e35f486040b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:28:49.702014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:49.702942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:49.703743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:28:49.703743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:28:49.703743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:28:49.703743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:28:49.703743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:28:49.703743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:28:49.703743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:49.703743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:49.713293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 208/414] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:49.715928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fc59a-91da-4324-a3e6-4e35f486040b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:49.746379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:49.838360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d0cf8-1fb1-477a-8427-acf30188c3a7 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:49.840731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:49.883613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fc59a-91da-4324-a3e6-4e35f486040b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:49.887270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed38f13-e887-4619-a706-412c5d182e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:49.887883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed38f13-e887-4619-a706-412c5d182e73 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:49.888160 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed38f13-e887-4619-a706-412c5d182e73 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:49.888483 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed38f13-e887-4619-a706-412c5d182e73 None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:49.889177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed38f13-e887-4619-a706-412c5d182e73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:49.889481 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed38f13-e887-4619-a706-412c5d182e73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:49.891539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed38f13-e887-4619-a706-412c5d182e73 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:49.907966 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 207/415] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:49.913968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed38f13-e887-4619-a706-412c5d182e73 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:49.960867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48695e09-f412-416c-8abf-4003aa358fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:49.961482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48695e09-f412-416c-8abf-4003aa358fe3 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:49.961756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48695e09-f412-416c-8abf-4003aa358fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:49.962023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48695e09-f412-416c-8abf-4003aa358fe3 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:49.962718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48695e09-f412-416c-8abf-4003aa358fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:49.963019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48695e09-f412-416c-8abf-4003aa358fe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:49.969815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48695e09-f412-416c-8abf-4003aa358fe3 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:50.000436 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 209/416] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:50.009323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48695e09-f412-416c-8abf-4003aa358fe3 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:50.042544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed38f13-e887-4619-a706-412c5d182e73 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:50.081729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfa3f6-abef-43a7-a37d-5b1876a5a51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:50.082397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfa3f6-abef-43a7-a37d-5b1876a5a51a None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:50.082917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfa3f6-abef-43a7-a37d-5b1876a5a51a None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:50.083199 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfa3f6-abef-43a7-a37d-5b1876a5a51a None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/70742ba8f3f6461f8f578b323afea7cc/roles/770887c5c808476d8505bf8b8fc5f27a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:50.101136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfa3f6-abef-43a7-a37d-5b1876a5a51a None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=70742ba8f3f6461f8f578b323afea7cc, role_id=770887c5c808476d8505bf8b8fc5f27a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:50.102604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfa3f6-abef-43a7-a37d-5b1876a5a51a None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:50.131107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 208/417] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:49 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/70742ba8f3f6461f8f578b323afea7cc/roles/770887c5c808476d8505bf8b8fc5f27a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:50.142124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:50.191747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48695e09-f412-416c-8abf-4003aa358fe3 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:50.236032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9b9d-9732-401f-ac7f-61311df9f1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:50.236758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9b9d-9732-401f-ac7f-61311df9f1bc None admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:50.237043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9b9d-9732-401f-ac7f-61311df9f1bc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:50.237330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9b9d-9732-401f-ac7f-61311df9f1bc None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/ae4b023dbf114bf98d0e38ea7fec94fa/roles/770887c5c808476d8505bf8b8fc5f27a` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:50.256762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce9b9d-9732-401f-ac7f-61311df9f1bc None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=ae4b023dbf114bf98d0e38ea7fec94fa, role_id=770887c5c808476d8505bf8b8fc5f27a)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:50.258270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce9b9d-9732-401f-ac7f-61311df9f1bc None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:50.281175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 210/418] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:50 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/ae4b023dbf114bf98d0e38ea7fec94fa/roles/770887c5c808476d8505bf8b8fc5f27a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:51.176091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab5e6-7ca3-4558-bc65-78215e3bbc4d None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:51.176636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab5e6-7ca3-4558-bc65-78215e3bbc4d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:51.176710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab5e6-7ca3-4558-bc65-78215e3bbc4d None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:51.178320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 209/419] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:51.198668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f82a9-a829-4e09-9984-ab624b1f2fa1 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:51.199734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f82a9-a829-4e09-9984-ab624b1f2fa1 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:51.199734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f82a9-a829-4e09-9984-ab624b1f2fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:51.220021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e53f82a9-a829-4e09-9984-ab624b1f2fa1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:51.225315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e53f82a9-a829-4e09-9984-ab624b1f2fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:51.313935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 211/420] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:51.785051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e7d47-9340-427b-b913-1e12ab924273 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:51.785478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e7d47-9340-427b-b913-1e12ab924273 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:51.785666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e7d47-9340-427b-b913-1e12ab924273 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:51.787212 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 210/421] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:51.797818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee202d-c3e1-4884-8334-a2597bc91cd8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:51.798107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee202d-c3e1-4884-8334-a2597bc91cd8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:51.798273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e7d47-9340-427b-b913-1e12ab924273 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:51.798426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee202d-c3e1-4884-8334-a2597bc91cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:51.814475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:51.821432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0bee202d-c3e1-4884-8334-a2597bc91cd8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:51.822640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0bee202d-c3e1-4884-8334-a2597bc91cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:51.888944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:51.911470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e7d47-9340-427b-b913-1e12ab924273 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:51.924182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 212/422] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:51.963669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036a68-188b-4dc7-84e9-a5641cf735aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:51.964533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036a68-188b-4dc7-84e9-a5641cf735aa None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:51.964766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036a68-188b-4dc7-84e9-a5641cf735aa None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:51.965036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036a68-188b-4dc7-84e9-a5641cf735aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:51.975434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14036a68-188b-4dc7-84e9-a5641cf735aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-14036a68-188b-4dc7-84e9-a5641cf735aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:51.984414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:51.985269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:51.986024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 211/423] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:51.991524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee202d-c3e1-4884-8334-a2597bc91cd8 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:52.131431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee202d-c3e1-4884-8334-a2597bc91cd8 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:52.174127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c62630-5cf2-4171-b859-16ea28ef5d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:52.175065 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c62630-5cf2-4171-b859-16ea28ef5d67 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:52.175561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c62630-5cf2-4171-b859-16ea28ef5d67 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:52.176028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c62630-5cf2-4171-b859-16ea28ef5d67 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:52.176910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c62630-5cf2-4171-b859-16ea28ef5d67 None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:52.177424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c62630-5cf2-4171-b859-16ea28ef5d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:52.179005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c62630-5cf2-4171-b859-16ea28ef5d67 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:52.188108 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 213/424] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:52.196372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036a68-188b-4dc7-84e9-a5641cf735aa None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:52.289705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c62630-5cf2-4171-b859-16ea28ef5d67 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:52.325387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:52.385129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036a68-188b-4dc7-84e9-a5641cf735aa None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:52.394960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:52.444690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19412ca6-3eb3-4f81-8d28-b76634029f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:52.446096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19412ca6-3eb3-4f81-8d28-b76634029f34 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:52.446986 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19412ca6-3eb3-4f81-8d28-b76634029f34 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:52.447576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19412ca6-3eb3-4f81-8d28-b76634029f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:52.461861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19412ca6-3eb3-4f81-8d28-b76634029f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:52.464923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19412ca6-3eb3-4f81-8d28-b76634029f34 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:52.475285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c62630-5cf2-4171-b859-16ea28ef5d67 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-19412ca6-3eb3-4f81-8d28-b76634029f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:52.482902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:52.483774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:52.484593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:52.484593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:52.484593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:52.484593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 212/425] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:52.492377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763365-c259-4484-86ca-87f7893c9bcf None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:52.492377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763365-c259-4484-86ca-87f7893c9bcf None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:52.492377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763365-c259-4484-86ca-87f7893c9bcf None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:52.492377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 213/426] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:52.492377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763365-c259-4484-86ca-87f7893c9bcf None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:52.522040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83d037-9af7-4e0d-ad96-3129ff63fdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:52.522666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83d037-9af7-4e0d-ad96-3129ff63fdcc None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:52.523221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83d037-9af7-4e0d-ad96-3129ff63fdcc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:52.523499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83d037-9af7-4e0d-ad96-3129ff63fdcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:52.534610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83d037-9af7-4e0d-ad96-3129ff63fdcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-df83d037-9af7-4e0d-ad96-3129ff63fdcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:52.540161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:52.541041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:52.541816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 214/427] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:52.543393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bda5a-4984-4545-a165-6496f548b8c1 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:52.543676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bda5a-4984-4545-a165-6496f548b8c1 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:52.543950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bda5a-4984-4545-a165-6496f548b8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:52.569438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-103bda5a-4984-4545-a165-6496f548b8c1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:52.570621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-103bda5a-4984-4545-a165-6496f548b8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:52.628459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763365-c259-4484-86ca-87f7893c9bcf None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:52.650250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 215/428] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:52.651172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bda5a-4984-4545-a165-6496f548b8c1 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:52.660160 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:52.676047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5853c-669a-4543-8008-08867c5f5fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:52.677552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5853c-669a-4543-8008-08867c5f5fc8 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:52.678056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5853c-669a-4543-8008-08867c5f5fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:52.678535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5853c-669a-4543-8008-08867c5f5fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:52.679890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5853c-669a-4543-8008-08867c5f5fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:52.680467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5853c-669a-4543-8008-08867c5f5fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:52.682538 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5853c-669a-4543-8008-08867c5f5fc8 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:52.697412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 214/429] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:52.704098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5853c-669a-4543-8008-08867c5f5fc8 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:52.735584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:52.820772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5853c-669a-4543-8008-08867c5f5fc8 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:52.843472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bda5a-4984-4545-a165-6496f548b8c1 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:52.858598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1df380-ee7c-4c7b-8944-d728fc65f886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:52.859533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1df380-ee7c-4c7b-8944-d728fc65f886 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:52.860027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1df380-ee7c-4c7b-8944-d728fc65f886 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:52.860514 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1df380-ee7c-4c7b-8944-d728fc65f886 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:52.869946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1df380-ee7c-4c7b-8944-d728fc65f886 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:52.872631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1df380-ee7c-4c7b-8944-d728fc65f886 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:52.882122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c5221-6cfa-456a-91d6-6d5a463ebb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:52.882727 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c5221-6cfa-456a-91d6-6d5a463ebb92 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:52.883008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c5221-6cfa-456a-91d6-6d5a463ebb92 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:52.883284 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c5221-6cfa-456a-91d6-6d5a463ebb92 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:52.883946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c5221-6cfa-456a-91d6-6d5a463ebb92 None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:52.884250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c5221-6cfa-456a-91d6-6d5a463ebb92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:52.886201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c5221-6cfa-456a-91d6-6d5a463ebb92 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-9e1df380-ee7c-4c7b-8944-d728fc65f886 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:52.888131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:52.889095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 14:28:52.890198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:52.892188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 215/430] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:28:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:28:52.896928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 216/431] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:52.898195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1df380-ee7c-4c7b-8944-d728fc65f886 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:52.898302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91493b-2de3-4de3-82bb-121caf1ef021 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:52.898474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91493b-2de3-4de3-82bb-121caf1ef021 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:52.898753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91493b-2de3-4de3-82bb-121caf1ef021 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:52.904423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 217/432] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:52.905284 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91493b-2de3-4de3-82bb-121caf1ef021 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:52.947716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:53.013039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1df380-ee7c-4c7b-8944-d728fc65f886 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.015887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:53.020901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91493b-2de3-4de3-82bb-121caf1ef021 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.045401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a8cae-1e70-4c0b-8208-d9352e76d9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.045999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a8cae-1e70-4c0b-8208-d9352e76d9e0 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.046272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a8cae-1e70-4c0b-8208-d9352e76d9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.046539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a8cae-1e70-4c0b-8208-d9352e76d9e0 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.047257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a8cae-1e70-4c0b-8208-d9352e76d9e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:53.047559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a8cae-1e70-4c0b-8208-d9352e76d9e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:53.049061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a8cae-1e70-4c0b-8208-d9352e76d9e0 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:53.055055 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd25cf-5e5a-47ef-8e2e-25eb95531a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.055718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd25cf-5e5a-47ef-8e2e-25eb95531a81 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.055999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd25cf-5e5a-47ef-8e2e-25eb95531a81 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.056272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd25cf-5e5a-47ef-8e2e-25eb95531a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.066529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 216/433] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Oct 20 14:28:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:53.067167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd25cf-5e5a-47ef-8e2e-25eb95531a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:53.067779 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0aa142-29d8-4fd6-9ee2-76d9d0c01002 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.068072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0aa142-29d8-4fd6-9ee2-76d9d0c01002 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.068353 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0aa142-29d8-4fd6-9ee2-76d9d0c01002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.068652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd25cf-5e5a-47ef-8e2e-25eb95531a81 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-c3fd25cf-5e5a-47ef-8e2e-25eb95531a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:53.083365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:53.084569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:53.085383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:53.085383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:53.085383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:53.085383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 218/434] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:53.085962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe7b20-e7ea-4751-b2a9-f4eaf5ac6e88 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.086376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe7b20-e7ea-4751-b2a9-f4eaf5ac6e88 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.086529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe7b20-e7ea-4751-b2a9-f4eaf5ac6e88 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.087034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5d0aa142-29d8-4fd6-9ee2-76d9d0c01002 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:53.092114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 219/435] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:53.092237 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5d0aa142-29d8-4fd6-9ee2-76d9d0c01002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:53.092334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe7b20-e7ea-4751-b2a9-f4eaf5ac6e88 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.155439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 217/436] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:53.156203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0aa142-29d8-4fd6-9ee2-76d9d0c01002 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.188866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:53.210090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe7b20-e7ea-4751-b2a9-f4eaf5ac6e88 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.245987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335031ef-e1ec-4c4a-8dda-ec49bdf912fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.247205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335031ef-e1ec-4c4a-8dda-ec49bdf912fc None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.247945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335031ef-e1ec-4c4a-8dda-ec49bdf912fc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.248453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335031ef-e1ec-4c4a-8dda-ec49bdf912fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.258942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335031ef-e1ec-4c4a-8dda-ec49bdf912fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:53.263464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0aa142-29d8-4fd6-9ee2-76d9d0c01002 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-335031ef-e1ec-4c4a-8dda-ec49bdf912fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:53.264302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:53.265182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:53.266038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 220/437] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:53.266682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335031ef-e1ec-4c4a-8dda-ec49bdf912fc None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.281942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:53.302098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618374a-a08b-42db-a089-3156339979ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.302728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618374a-a08b-42db-a089-3156339979ee None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.303009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618374a-a08b-42db-a089-3156339979ee None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.303290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618374a-a08b-42db-a089-3156339979ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.314289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618374a-a08b-42db-a089-3156339979ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:53.315800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618374a-a08b-42db-a089-3156339979ee None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-3618374a-a08b-42db-a089-3156339979ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:53.331029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:53.331952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:53.332837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:53.332837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:53.332837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:53.332837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 218/438] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:53.333553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736aab9-cd39-4914-a62b-3df9eb2f34ed None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.333652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736aab9-cd39-4914-a62b-3df9eb2f34ed None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.333895 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736aab9-cd39-4914-a62b-3df9eb2f34ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.357448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-9736aab9-cd39-4914-a62b-3df9eb2f34ed None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:53.358656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9736aab9-cd39-4914-a62b-3df9eb2f34ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:53.389694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335031ef-e1ec-4c4a-8dda-ec49bdf912fc None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.428379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 219/439] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:53.429148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736aab9-cd39-4914-a62b-3df9eb2f34ed None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.433245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41b9be-e829-4fff-b30e-23666cbc6e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.433864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41b9be-e829-4fff-b30e-23666cbc6e63 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.434131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41b9be-e829-4fff-b30e-23666cbc6e63 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.434402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41b9be-e829-4fff-b30e-23666cbc6e63 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.435043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41b9be-e829-4fff-b30e-23666cbc6e63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:53.435343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41b9be-e829-4fff-b30e-23666cbc6e63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:53.436743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41b9be-e829-4fff-b30e-23666cbc6e63 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:53.446231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:53.453870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 221/440] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:53.453870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41b9be-e829-4fff-b30e-23666cbc6e63 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.523471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:53.544250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736aab9-cd39-4914-a62b-3df9eb2f34ed None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.558642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41b9be-e829-4fff-b30e-23666cbc6e63 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.582985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e94ca0-06ac-468c-a029-22fc5261c1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.583596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e94ca0-06ac-468c-a029-22fc5261c1aa None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.583877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e94ca0-06ac-468c-a029-22fc5261c1aa None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.584152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e94ca0-06ac-468c-a029-22fc5261c1aa None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.584846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e94ca0-06ac-468c-a029-22fc5261c1aa None admin] RBAC: Adding query filter params (name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:53.585151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e94ca0-06ac-468c-a029-22fc5261c1aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:53.586526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e94ca0-06ac-468c-a029-22fc5261c1aa None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:53.594317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855dd88-1422-4eb5-9da3-a498754fac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.594826 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 220/441] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:53.595523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e94ca0-06ac-468c-a029-22fc5261c1aa None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.595731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855dd88-1422-4eb5-9da3-a498754fac2c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.596180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855dd88-1422-4eb5-9da3-a498754fac2c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.596491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855dd88-1422-4eb5-9da3-a498754fac2c None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.597131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855dd88-1422-4eb5-9da3-a498754fac2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:53.597437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855dd88-1422-4eb5-9da3-a498754fac2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:53.598800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855dd88-1422-4eb5-9da3-a498754fac2c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:53.611882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 222/442] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:53.612778 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855dd88-1422-4eb5-9da3-a498754fac2c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.677000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:53.701010 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e94ca0-06ac-468c-a029-22fc5261c1aa None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.723453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855dd88-1422-4eb5-9da3-a498754fac2c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.734310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bbabd-1d75-463c-8c86-44fdbe45d364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.734936 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bbabd-1d75-463c-8c86-44fdbe45d364 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.735215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bbabd-1d75-463c-8c86-44fdbe45d364 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.735490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bbabd-1d75-463c-8c86-44fdbe45d364 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.745598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bbabd-1d75-463c-8c86-44fdbe45d364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-556bbabd-1d75-463c-8c86-44fdbe45d364 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:53.750357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:53.751290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:53.752069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 221/443] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:53.752749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bbabd-1d75-463c-8c86-44fdbe45d364 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.753654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:53.759776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a680343-b7a0-48f7-9557-d8da2070b92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.760434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680343-b7a0-48f7-9557-d8da2070b92a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.760744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680343-b7a0-48f7-9557-d8da2070b92a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.761021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680343-b7a0-48f7-9557-d8da2070b92a None admin] PATH_INFO: `/v3/users/nova` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.768072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a680343-b7a0-48f7-9557-d8da2070b92a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:53.769582 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a680343-b7a0-48f7-9557-d8da2070b92a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-2a680343-b7a0-48f7-9557-d8da2070b92a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:53.781418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:53.782458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 223/444] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:53.783538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a680343-b7a0-48f7-9557-d8da2070b92a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.855289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bbabd-1d75-463c-8c86-44fdbe45d364 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.881961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a680343-b7a0-48f7-9557-d8da2070b92a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:53.890376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11926077-ebbc-4fa2-b415-3f4d57d7fdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.891003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11926077-ebbc-4fa2-b415-3f4d57d7fdb1 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.891283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11926077-ebbc-4fa2-b415-3f4d57d7fdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.891561 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11926077-ebbc-4fa2-b415-3f4d57d7fdb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.894180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:53.902418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11926077-ebbc-4fa2-b415-3f4d57d7fdb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:53.904047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11926077-ebbc-4fa2-b415-3f4d57d7fdb1 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:53.914443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7a19c-18c3-478d-9d62-4ac0de061e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:53.915058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7a19c-18c3-478d-9d62-4ac0de061e34 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:53.915334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7a19c-18c3-478d-9d62-4ac0de061e34 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:53.915627 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7a19c-18c3-478d-9d62-4ac0de061e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-11926077-ebbc-4fa2-b415-3f4d57d7fdb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:53.918711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:53.919798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:53.920653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:53.920653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:53.920653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:53.920653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 222/445] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:53.920653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11926077-ebbc-4fa2-b415-3f4d57d7fdb1 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.922759 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7a19c-18c3-478d-9d62-4ac0de061e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:53.924227 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7a19c-18c3-478d-9d62-4ac0de061e34 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-4fa7a19c-18c3-478d-9d62-4ac0de061e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:28:53.936076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:53.937193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 224/446] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:53.938231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7a19c-18c3-478d-9d62-4ac0de061e34 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:53.989885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:54.024732 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11926077-ebbc-4fa2-b415-3f4d57d7fdb1 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.042245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7a19c-18c3-478d-9d62-4ac0de061e34 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.057917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84783f08-8246-45ec-a960-8f7947f0c17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.058550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783f08-8246-45ec-a960-8f7947f0c17c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.058837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783f08-8246-45ec-a960-8f7947f0c17c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.059112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783f08-8246-45ec-a960-8f7947f0c17c None admin] PATH_INFO: `/v3/roles/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.069001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84783f08-8246-45ec-a960-8f7947f0c17c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-84783f08-8246-45ec-a960-8f7947f0c17c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:54.074872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:54.075922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:54.076737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:54.077318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69186217-5681-48a4-b7de-b4cf49c2e81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.077318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69186217-5681-48a4-b7de-b4cf49c2e81d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.077450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 223/447] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:54.077512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69186217-5681-48a4-b7de-b4cf49c2e81d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.077512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69186217-5681-48a4-b7de-b4cf49c2e81d None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.077635 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84783f08-8246-45ec-a960-8f7947f0c17c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.077697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69186217-5681-48a4-b7de-b4cf49c2e81d None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:54.077697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69186217-5681-48a4-b7de-b4cf49c2e81d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.078820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69186217-5681-48a4-b7de-b4cf49c2e81d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.092191 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 225/448] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:54.092191 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69186217-5681-48a4-b7de-b4cf49c2e81d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.131424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:54.181541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84783f08-8246-45ec-a960-8f7947f0c17c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.185648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69186217-5681-48a4-b7de-b4cf49c2e81d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.197631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:54.215641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc75d2-1dc8-469c-bb32-5dfabf93a4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.216245 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc75d2-1dc8-469c-bb32-5dfabf93a4c2 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.216681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc75d2-1dc8-469c-bb32-5dfabf93a4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.216961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc75d2-1dc8-469c-bb32-5dfabf93a4c2 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.217688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc75d2-1dc8-469c-bb32-5dfabf93a4c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:54.217998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc75d2-1dc8-469c-bb32-5dfabf93a4c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.219345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b301b6c-b5f7-4649-8191-376251ddea89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.219453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc75d2-1dc8-469c-bb32-5dfabf93a4c2 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.220045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b301b6c-b5f7-4649-8191-376251ddea89 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.220321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b301b6c-b5f7-4649-8191-376251ddea89 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.220621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b301b6c-b5f7-4649-8191-376251ddea89 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.221249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b301b6c-b5f7-4649-8191-376251ddea89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:54.221547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b301b6c-b5f7-4649-8191-376251ddea89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.222900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b301b6c-b5f7-4649-8191-376251ddea89 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.235749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 226/449] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:54.236565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b301b6c-b5f7-4649-8191-376251ddea89 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.236947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 224/450] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:54.237764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc75d2-1dc8-469c-bb32-5dfabf93a4c2 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.334670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b301b6c-b5f7-4649-8191-376251ddea89 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.335536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc75d2-1dc8-469c-bb32-5dfabf93a4c2 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.343014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:54.371583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a93c6-bbd9-46cd-9966-41b3be1d40fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.372190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a93c6-bbd9-46cd-9966-41b3be1d40fc None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.372507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a93c6-bbd9-46cd-9966-41b3be1d40fc None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.372787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a93c6-bbd9-46cd-9966-41b3be1d40fc None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.372876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd61273-bffd-4c43-aa3d-195643ae4929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.373415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd61273-bffd-4c43-aa3d-195643ae4929 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.373508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a93c6-bbd9-46cd-9966-41b3be1d40fc None admin] RBAC: Adding query filter params (name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:54.373697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd61273-bffd-4c43-aa3d-195643ae4929 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.373791 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a93c6-bbd9-46cd-9966-41b3be1d40fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.373972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd61273-bffd-4c43-aa3d-195643ae4929 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.374653 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd61273-bffd-4c43-aa3d-195643ae4929 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:54.374961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd61273-bffd-4c43-aa3d-195643ae4929 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.375125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a93c6-bbd9-46cd-9966-41b3be1d40fc None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.376395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd61273-bffd-4c43-aa3d-195643ae4929 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.382780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 225/451] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:54.383556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a93c6-bbd9-46cd-9966-41b3be1d40fc None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.387756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 227/452] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:54.387756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd61273-bffd-4c43-aa3d-195643ae4929 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.421965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:54.480827 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a93c6-bbd9-46cd-9966-41b3be1d40fc None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.489243 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd61273-bffd-4c43-aa3d-195643ae4929 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.515888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237ba6f-0111-4dbb-9257-a738aecd8f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.516544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237ba6f-0111-4dbb-9257-a738aecd8f7e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.516832 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237ba6f-0111-4dbb-9257-a738aecd8f7e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.517106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237ba6f-0111-4dbb-9257-a738aecd8f7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.522192 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ffee5-a07a-41aa-9ca3-6b1d02bf70be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.522819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ffee5-a07a-41aa-9ca3-6b1d02bf70be None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.523099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ffee5-a07a-41aa-9ca3-6b1d02bf70be None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.523373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ffee5-a07a-41aa-9ca3-6b1d02bf70be None admin] PATH_INFO: `/v3/users/glance` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.527783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237ba6f-0111-4dbb-9257-a738aecd8f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.529256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237ba6f-0111-4dbb-9257-a738aecd8f7e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.531040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ffee5-a07a-41aa-9ca3-6b1d02bf70be None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.532497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ffee5-a07a-41aa-9ca3-6b1d02bf70be None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-3237ba6f-0111-4dbb-9257-a738aecd8f7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:54.543277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:54.544279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:54.545150 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:54.545150 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:54.545150 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-791ffee5-a07a-41aa-9ca3-6b1d02bf70be None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:54.545306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:54.546158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 228/454] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:54.547014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ffee5-a07a-41aa-9ca3-6b1d02bf70be None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.548942 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 226/454] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:54.549722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237ba6f-0111-4dbb-9257-a738aecd8f7e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.571874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:54.637215 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:54.646107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ffee5-a07a-41aa-9ca3-6b1d02bf70be None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.649245 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237ba6f-0111-4dbb-9257-a738aecd8f7e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.679247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e44129-e674-493c-94e9-6ed3db33c409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.679953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e44129-e674-493c-94e9-6ed3db33c409 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.680301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e44129-e674-493c-94e9-6ed3db33c409 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.680669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e44129-e674-493c-94e9-6ed3db33c409 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.681144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819b41a-12e3-4f58-9153-0fd79f29f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.681763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819b41a-12e3-4f58-9153-0fd79f29f0a4 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.682035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819b41a-12e3-4f58-9153-0fd79f29f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.682301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819b41a-12e3-4f58-9153-0fd79f29f0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.693620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819b41a-12e3-4f58-9153-0fd79f29f0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.695200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819b41a-12e3-4f58-9153-0fd79f29f0a4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-d819b41a-12e3-4f58-9153-0fd79f29f0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:54.709563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:54.710439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:54.711262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:54.711262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:54.711262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:54.711262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 227/455] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:54.711477 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819b41a-12e3-4f58-9153-0fd79f29f0a4 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.751587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e44129-e674-493c-94e9-6ed3db33c409 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.753080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e44129-e674-493c-94e9-6ed3db33c409 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.780172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:54.809827 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819b41a-12e3-4f58-9153-0fd79f29f0a4 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.841556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:54.843404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03c409-8570-49d8-a0d1-e1f450b94ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.844179 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03c409-8570-49d8-a0d1-e1f450b94ce2 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.844494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03c409-8570-49d8-a0d1-e1f450b94ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.844786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03c409-8570-49d8-a0d1-e1f450b94ce2 None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/61d6c1cd010d4fa387fa64864419bc6a/roles/770887c5c808476d8505bf8b8fc5f27a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.851553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 229/456] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:54.853243 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e44129-e674-493c-94e9-6ed3db33c409 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.862518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03c409-8570-49d8-a0d1-e1f450b94ce2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=61d6c1cd010d4fa387fa64864419bc6a, role_id=770887c5c808476d8505bf8b8fc5f27a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:54.863967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03c409-8570-49d8-a0d1-e1f450b94ce2 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:54.876617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 228/457] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:28:54 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/61d6c1cd010d4fa387fa64864419bc6a/roles/770887c5c808476d8505bf8b8fc5f27a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:28:54.877397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03c409-8570-49d8-a0d1-e1f450b94ce2 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:54.958523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e44129-e674-493c-94e9-6ed3db33c409 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.993530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03c409-8570-49d8-a0d1-e1f450b94ce2 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:54.997925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c0fc7-490e-4bb3-8205-1c1a6324e265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:54.998548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c0fc7-490e-4bb3-8205-1c1a6324e265 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:54.998827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c0fc7-490e-4bb3-8205-1c1a6324e265 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:54.999100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c0fc7-490e-4bb3-8205-1c1a6324e265 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:54.999751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c0fc7-490e-4bb3-8205-1c1a6324e265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:55.000178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c0fc7-490e-4bb3-8205-1c1a6324e265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.002801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c0fc7-490e-4bb3-8205-1c1a6324e265 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.018332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 230/458] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:55.020402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c0fc7-490e-4bb3-8205-1c1a6324e265 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.022248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:55.037634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8b5e7-3052-4f51-9a98-0b898d283fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.038438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8b5e7-3052-4f51-9a98-0b898d283fcf None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.038725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8b5e7-3052-4f51-9a98-0b898d283fcf None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.038999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8b5e7-3052-4f51-9a98-0b898d283fcf None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.039792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8b5e7-3052-4f51-9a98-0b898d283fcf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:55.040106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8b5e7-3052-4f51-9a98-0b898d283fcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.041598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8b5e7-3052-4f51-9a98-0b898d283fcf None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.061554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 229/459] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Oct 20 14:28:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:55.062401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8b5e7-3052-4f51-9a98-0b898d283fcf None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.086645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:55.140785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c0fc7-490e-4bb3-8205-1c1a6324e265 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:55.176079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c950066-e830-4fe9-9c0a-693792f1760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.176707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c950066-e830-4fe9-9c0a-693792f1760b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.176986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c950066-e830-4fe9-9c0a-693792f1760b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.177259 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c950066-e830-4fe9-9c0a-693792f1760b None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.177674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8b5e7-3052-4f51-9a98-0b898d283fcf None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:55.177895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c950066-e830-4fe9-9c0a-693792f1760b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:55.178197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c950066-e830-4fe9-9c0a-693792f1760b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.179561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c950066-e830-4fe9-9c0a-693792f1760b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.196363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 231/460] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:55.196363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c950066-e830-4fe9-9c0a-693792f1760b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.222149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9374ea7-a76a-4a17-9854-35aa3fae8549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.222702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9374ea7-a76a-4a17-9854-35aa3fae8549 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.222991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9374ea7-a76a-4a17-9854-35aa3fae8549 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.223248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9374ea7-a76a-4a17-9854-35aa3fae8549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.234625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9374ea7-a76a-4a17-9854-35aa3fae8549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.236097 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9374ea7-a76a-4a17-9854-35aa3fae8549 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-e9374ea7-a76a-4a17-9854-35aa3fae8549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:55.252226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:55.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:55.254014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:55.254014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:55.254014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:55.254014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 230/461] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:55.254226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9374ea7-a76a-4a17-9854-35aa3fae8549 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.287613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:55.304725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c950066-e830-4fe9-9c0a-693792f1760b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:55.319082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:55.341402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838ec2e-af49-4085-a5d5-62c80a86ebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.342032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838ec2e-af49-4085-a5d5-62c80a86ebab None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.342308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838ec2e-af49-4085-a5d5-62c80a86ebab None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.342714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838ec2e-af49-4085-a5d5-62c80a86ebab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.350859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838ec2e-af49-4085-a5d5-62c80a86ebab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.351259 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838ec2e-af49-4085-a5d5-62c80a86ebab None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-a838ec2e-af49-4085-a5d5-62c80a86ebab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:55.363361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:55.364266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 14:28:55.365165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:55.369291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 232/462] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:55.369395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9374ea7-a76a-4a17-9854-35aa3fae8549 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:55.370517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838ec2e-af49-4085-a5d5-62c80a86ebab None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.404932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e8f8e-9270-4797-b0f7-3136301f7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.405648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8f8e-9270-4797-b0f7-3136301f7137 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.405995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8f8e-9270-4797-b0f7-3136301f7137 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.406347 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8f8e-9270-4797-b0f7-3136301f7137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.477632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e8f8e-9270-4797-b0f7-3136301f7137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.478502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838ec2e-af49-4085-a5d5-62c80a86ebab None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:55.479026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e8f8e-9270-4797-b0f7-3136301f7137 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.510189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:55.515561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2fc28-cce9-4549-9dc0-14e15b3c22cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.516196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2fc28-cce9-4549-9dc0-14e15b3c22cb None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.516495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2fc28-cce9-4549-9dc0-14e15b3c22cb None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.516764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2fc28-cce9-4549-9dc0-14e15b3c22cb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.525183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2fc28-cce9-4549-9dc0-14e15b3c22cb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.527218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2fc28-cce9-4549-9dc0-14e15b3c22cb None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-c6c2fc28-cce9-4549-9dc0-14e15b3c22cb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:55.539350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:55.540233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:55.541148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 233/463] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:55.541692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2fc28-cce9-4549-9dc0-14e15b3c22cb None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.574663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:55.589395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 231/464] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:55.589395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e8f8e-9270-4797-b0f7-3136301f7137 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.657362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2fc28-cce9-4549-9dc0-14e15b3c22cb None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:55.690858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e8f8e-9270-4797-b0f7-3136301f7137 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:55.696323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3bef2-c32f-4bd0-9657-c66a6a189b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.697126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3bef2-c32f-4bd0-9657-c66a6a189b4d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.697547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3bef2-c32f-4bd0-9657-c66a6a189b4d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.697961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3bef2-c32f-4bd0-9657-c66a6a189b4d None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.699765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3bef2-c32f-4bd0-9657-c66a6a189b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:55.700217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3bef2-c32f-4bd0-9657-c66a6a189b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.702433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3bef2-c32f-4bd0-9657-c66a6a189b4d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.722567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:55.722876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 234/465] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:55.724191 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3bef2-c32f-4bd0-9657-c66a6a189b4d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.736113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2312d-606f-4358-9b1e-4b5b97c7fbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.736680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2312d-606f-4358-9b1e-4b5b97c7fbcb None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.736957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2312d-606f-4358-9b1e-4b5b97c7fbcb None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.737234 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2312d-606f-4358-9b1e-4b5b97c7fbcb None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.738688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2312d-606f-4358-9b1e-4b5b97c7fbcb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:55.739001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2312d-606f-4358-9b1e-4b5b97c7fbcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.740784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2312d-606f-4358-9b1e-4b5b97c7fbcb None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.769056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 232/466] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:55.769946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2312d-606f-4358-9b1e-4b5b97c7fbcb None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.841094 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:55.850319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3bef2-c32f-4bd0-9657-c66a6a189b4d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:55.886603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2312d-606f-4358-9b1e-4b5b97c7fbcb None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:55.892522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c20f3-155c-4e41-a843-ae5132b4b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.893163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c20f3-155c-4e41-a843-ae5132b4b688 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.893445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c20f3-155c-4e41-a843-ae5132b4b688 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.893719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c20f3-155c-4e41-a843-ae5132b4b688 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.894447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c20f3-155c-4e41-a843-ae5132b4b688 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:55.894754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c20f3-155c-4e41-a843-ae5132b4b688 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.897110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c20f3-155c-4e41-a843-ae5132b4b688 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.920719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 235/467] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:55.921935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c20f3-155c-4e41-a843-ae5132b4b688 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:55.932329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe1f3a-389e-449b-bade-fd39468abf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:55.934398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe1f3a-389e-449b-bade-fd39468abf8b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:55.934854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe1f3a-389e-449b-bade-fd39468abf8b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:55.935295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe1f3a-389e-449b-bade-fd39468abf8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:55.948307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe1f3a-389e-449b-bade-fd39468abf8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:55.950602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe1f3a-389e-449b-bade-fd39468abf8b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-2bbe1f3a-389e-449b-bade-fd39468abf8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:55.969210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:55.970203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:55.971013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:55.971013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:55.971013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:55.971013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 233/468] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:55.971737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe1f3a-389e-449b-bade-fd39468abf8b None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:56.022791 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:56.048539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c20f3-155c-4e41-a843-ae5132b4b688 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:56.090711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe1f3a-389e-449b-bade-fd39468abf8b None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:56.094847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc9372-3884-4699-baca-cd12f4aa14d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:56.095493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc9372-3884-4699-baca-cd12f4aa14d4 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:56.095803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc9372-3884-4699-baca-cd12f4aa14d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:56.096060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc9372-3884-4699-baca-cd12f4aa14d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:56.103314 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc9372-3884-4699-baca-cd12f4aa14d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:56.105439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc9372-3884-4699-baca-cd12f4aa14d4 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-bafc9372-3884-4699-baca-cd12f4aa14d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:28:56.119961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:56.121019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:56.121851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:28:56.121851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:28:56.121851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:28:56.121851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:28:56.121851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:28:56.121851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:28:56.121851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:56.121851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:56.123901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 236/469] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:56.124927 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc9372-3884-4699-baca-cd12f4aa14d4 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:56.136995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372bc36-4c8a-402a-8aae-e7f364775e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:56.137980 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:56.138323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372bc36-4c8a-402a-8aae-e7f364775e17 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:56.138642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372bc36-4c8a-402a-8aae-e7f364775e17 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:56.138926 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372bc36-4c8a-402a-8aae-e7f364775e17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:56.234931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372bc36-4c8a-402a-8aae-e7f364775e17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:56.237071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372bc36-4c8a-402a-8aae-e7f364775e17 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:56.242850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc9372-3884-4699-baca-cd12f4aa14d4 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:56.286042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:56.291794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942779f-5405-4382-9b8e-3c41e19d8777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:56.292728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942779f-5405-4382-9b8e-3c41e19d8777 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:56.293148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942779f-5405-4382-9b8e-3c41e19d8777 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:56.293659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942779f-5405-4382-9b8e-3c41e19d8777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:56.363142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 234/470] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:56.363753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372bc36-4c8a-402a-8aae-e7f364775e17 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:56.391492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942779f-5405-4382-9b8e-3c41e19d8777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:56.392994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942779f-5405-4382-9b8e-3c41e19d8777 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:56.404756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:56.482845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372bc36-4c8a-402a-8aae-e7f364775e17 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:56.516428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 237/471] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:56.516428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942779f-5405-4382-9b8e-3c41e19d8777 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:56.528290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccf280-c84d-4f67-a6f1-8c97e3308a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:56.529143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccf280-c84d-4f67-a6f1-8c97e3308a51 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:56.529425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccf280-c84d-4f67-a6f1-8c97e3308a51 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:56.529701 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccf280-c84d-4f67-a6f1-8c97e3308a51 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:56.530351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccf280-c84d-4f67-a6f1-8c97e3308a51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:56.530656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccf280-c84d-4f67-a6f1-8c97e3308a51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:56.532167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccf280-c84d-4f67-a6f1-8c97e3308a51 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:56.550983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 235/472] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:56.550983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccf280-c84d-4f67-a6f1-8c97e3308a51 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:56.551199 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:56.636855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942779f-5405-4382-9b8e-3c41e19d8777 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:56.647880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:56.656967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccf280-c84d-4f67-a6f1-8c97e3308a51 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:56.672264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d1f6d-c4d2-4d92-91f0-85097d8d3850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:56.672895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d1f6d-c4d2-4d92-91f0-85097d8d3850 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:56.673188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d1f6d-c4d2-4d92-91f0-85097d8d3850 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:56.673462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d1f6d-c4d2-4d92-91f0-85097d8d3850 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:56.674156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d1f6d-c4d2-4d92-91f0-85097d8d3850 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:56.674463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d1f6d-c4d2-4d92-91f0-85097d8d3850 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:56.676063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d1f6d-c4d2-4d92-91f0-85097d8d3850 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:56.689471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 238/473] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:56.690283 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d1f6d-c4d2-4d92-91f0-85097d8d3850 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:56.701415 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e17c7-bf2a-4b58-b538-540f60d6c1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:56.702061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e17c7-bf2a-4b58-b538-540f60d6c1c2 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:56.702341 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e17c7-bf2a-4b58-b538-540f60d6c1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:56.702613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e17c7-bf2a-4b58-b538-540f60d6c1c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:56.715152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e17c7-bf2a-4b58-b538-540f60d6c1c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:56.716677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e17c7-bf2a-4b58-b538-540f60d6c1c2 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-ac7e17c7-bf2a-4b58-b538-540f60d6c1c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:56.731465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:56.732419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:56.738504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:56.738504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:56.738504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:56.738504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 236/474] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:56.738504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e17c7-bf2a-4b58-b538-540f60d6c1c2 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:56.795450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:56.804046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d1f6d-c4d2-4d92-91f0-85097d8d3850 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:56.840014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cc186-751f-4683-a19f-6df8ec9c6761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:56.840675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cc186-751f-4683-a19f-6df8ec9c6761 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:56.840958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cc186-751f-4683-a19f-6df8ec9c6761 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:56.841240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cc186-751f-4683-a19f-6df8ec9c6761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:56.843555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e17c7-bf2a-4b58-b538-540f60d6c1c2 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:56.852327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354cc186-751f-4683-a19f-6df8ec9c6761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:56.853828 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354cc186-751f-4683-a19f-6df8ec9c6761 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-354cc186-751f-4683-a19f-6df8ec9c6761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:56.868931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:56.869833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:56.870671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:56.870671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:56.870671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:56.870671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 239/475] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:56.870877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cc186-751f-4683-a19f-6df8ec9c6761 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:56.887649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be147fc5-9302-45b2-96b2-a3e2a2f7edcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:56.888278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be147fc5-9302-45b2-96b2-a3e2a2f7edcc None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:56.888590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be147fc5-9302-45b2-96b2-a3e2a2f7edcc None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:56.888863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be147fc5-9302-45b2-96b2-a3e2a2f7edcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:56.895629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be147fc5-9302-45b2-96b2-a3e2a2f7edcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:56.897115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be147fc5-9302-45b2-96b2-a3e2a2f7edcc None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:56.899541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-be147fc5-9302-45b2-96b2-a3e2a2f7edcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:28:56.910226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:56.911101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:56.911930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:28:56.911930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:28:56.911930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:28:56.911930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:28:56.911930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:28:56.911930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:28:56.911930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:56.911930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:56.911930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 237/476] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:56.912949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be147fc5-9302-45b2-96b2-a3e2a2f7edcc None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:56.997369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cc186-751f-4683-a19f-6df8ec9c6761 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.022814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be147fc5-9302-45b2-96b2-a3e2a2f7edcc None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.036188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769565a8-9983-4665-82a1-f237277527b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.037028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769565a8-9983-4665-82a1-f237277527b7 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.037313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769565a8-9983-4665-82a1-f237277527b7 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.037741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:57.037985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769565a8-9983-4665-82a1-f237277527b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.039043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769565a8-9983-4665-82a1-f237277527b7 None admin] RBAC: Adding query filter params (role.id=770887c5c808476d8505bf8b8fc5f27a, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=9647f8caa6cd4208800e2371fdf28301) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:57.039400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769565a8-9983-4665-82a1-f237277527b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.041262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769565a8-9983-4665-82a1-f237277527b7 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.060474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 240/477] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:56 2022] GET /identity/v3/role_assignments?user.id=9647f8caa6cd4208800e2371fdf28301&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=770887c5c808476d8505bf8b8fc5f27a => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:57.060474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769565a8-9983-4665-82a1-f237277527b7 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.072804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4026d-ea65-4db9-971a-b66b2fb94f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.073631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4026d-ea65-4db9-971a-b66b2fb94f31 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.074018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4026d-ea65-4db9-971a-b66b2fb94f31 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.075051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4026d-ea65-4db9-971a-b66b2fb94f31 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.075733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4026d-ea65-4db9-971a-b66b2fb94f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:57.076336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4026d-ea65-4db9-971a-b66b2fb94f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.078015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4026d-ea65-4db9-971a-b66b2fb94f31 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.096570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 238/478] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:57.101439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4026d-ea65-4db9-971a-b66b2fb94f31 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.151044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:57.168800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769565a8-9983-4665-82a1-f237277527b7 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.206457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dbd31-2b60-401a-9fec-eddb2615b34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.207080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dbd31-2b60-401a-9fec-eddb2615b34c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.207364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dbd31-2b60-401a-9fec-eddb2615b34c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.207643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dbd31-2b60-401a-9fec-eddb2615b34c None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.208294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dbd31-2b60-401a-9fec-eddb2615b34c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:57.208645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dbd31-2b60-401a-9fec-eddb2615b34c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.210082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dbd31-2b60-401a-9fec-eddb2615b34c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.215459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4026d-ea65-4db9-971a-b66b2fb94f31 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.224038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 241/479] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:57.225432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7348c-7e0b-4ad8-aff9-a7a6af0adc91 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.225720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7348c-7e0b-4ad8-aff9-a7a6af0adc91 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.226638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7348c-7e0b-4ad8-aff9-a7a6af0adc91 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.232672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 242/480] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:28:57.232672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7348c-7e0b-4ad8-aff9-a7a6af0adc91 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.261412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d67876-b808-4c6c-85a2-d283560eb9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.262023 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d67876-b808-4c6c-85a2-d283560eb9d0 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.262599 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d67876-b808-4c6c-85a2-d283560eb9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.262599 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d67876-b808-4c6c-85a2-d283560eb9d0 None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.263250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d67876-b808-4c6c-85a2-d283560eb9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:57.264044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d67876-b808-4c6c-85a2-d283560eb9d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.265043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d67876-b808-4c6c-85a2-d283560eb9d0 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.278416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 239/481] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:57.283394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d67876-b808-4c6c-85a2-d283560eb9d0 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.318322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:57.347530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7348c-7e0b-4ad8-aff9-a7a6af0adc91 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.384962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d26e1-e9e2-4b92-a164-2ce659410afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.386330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d26e1-e9e2-4b92-a164-2ce659410afb None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.386589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d26e1-e9e2-4b92-a164-2ce659410afb None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.386864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d26e1-e9e2-4b92-a164-2ce659410afb None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.392079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d67876-b808-4c6c-85a2-d283560eb9d0 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.393986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d26e1-e9e2-4b92-a164-2ce659410afb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.395574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d26e1-e9e2-4b92-a164-2ce659410afb None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-560d26e1-e9e2-4b92-a164-2ce659410afb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:28:57.408379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:57.409270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:57.410077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:28:57.410077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:28:57.410077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:28:57.410077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:28:57.410077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:28:57.410077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:28:57.410077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:57.410077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:57.410077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 243/482] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:57.411068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6b795-48f6-4e8f-a27b-0af1084253fd None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.411068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6b795-48f6-4e8f-a27b-0af1084253fd None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.411304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6b795-48f6-4e8f-a27b-0af1084253fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.424646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:57.431314 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-3ee6b795-48f6-4e8f-a27b-0af1084253fd None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:57.432102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e05e6-c0e6-43a1-a5a7-ab91d17bad85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.432648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3ee6b795-48f6-4e8f-a27b-0af1084253fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:57.432837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e05e6-c0e6-43a1-a5a7-ab91d17bad85 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.433120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e05e6-c0e6-43a1-a5a7-ab91d17bad85 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.433395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e05e6-c0e6-43a1-a5a7-ab91d17bad85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.440095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e05e6-c0e6-43a1-a5a7-ab91d17bad85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.441594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e05e6-c0e6-43a1-a5a7-ab91d17bad85 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-995e05e6-c0e6-43a1-a5a7-ab91d17bad85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:28:57.453554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:57.454429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:57.455243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:28:57.455243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:28:57.455243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:28:57.455243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:28:57.455243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:28:57.455243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:28:57.455243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:28:57.455243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:57.455243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 240/483] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:57.455705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e05e6-c0e6-43a1-a5a7-ab91d17bad85 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.512676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 244/484] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:28:57.512676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6b795-48f6-4e8f-a27b-0af1084253fd None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.560264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:57.573396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e05e6-c0e6-43a1-a5a7-ab91d17bad85 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.614917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46886d50-77d9-4e0f-b8f9-40171a9bd2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.615936 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886d50-77d9-4e0f-b8f9-40171a9bd2dd None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.616505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886d50-77d9-4e0f-b8f9-40171a9bd2dd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.616929 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886d50-77d9-4e0f-b8f9-40171a9bd2dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.618241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46886d50-77d9-4e0f-b8f9-40171a9bd2dd None admin] RBAC: Adding query filter params (role.id=770887c5c808476d8505bf8b8fc5f27a, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=cacfce5c9d07471cbe1713a4d6aa29f0) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:57.619030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46886d50-77d9-4e0f-b8f9-40171a9bd2dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.621065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46886d50-77d9-4e0f-b8f9-40171a9bd2dd None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.628975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 241/485] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/role_assignments?user.id=cacfce5c9d07471cbe1713a4d6aa29f0&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=770887c5c808476d8505bf8b8fc5f27a => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:57.630006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46886d50-77d9-4e0f-b8f9-40171a9bd2dd None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.630179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6b795-48f6-4e8f-a27b-0af1084253fd None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.680862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e6b6a-2e09-450e-958a-5f5445e67bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.687881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e6b6a-2e09-450e-958a-5f5445e67bfe None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.688277 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e6b6a-2e09-450e-958a-5f5445e67bfe None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.688629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e6b6a-2e09-450e-958a-5f5445e67bfe None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.689380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e6b6a-2e09-450e-958a-5f5445e67bfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:57.689690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e6b6a-2e09-450e-958a-5f5445e67bfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.691247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e6b6a-2e09-450e-958a-5f5445e67bfe None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.694324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:57.707110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 245/486] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:57.726898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e6b6a-2e09-450e-958a-5f5445e67bfe None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.843416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e6b6a-2e09-450e-958a-5f5445e67bfe None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.852613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46886d50-77d9-4e0f-b8f9-40171a9bd2dd None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:57.895639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd905b-50bf-4f13-a4b3-7a2f90667358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.895859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd905b-50bf-4f13-a4b3-7a2f90667358 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.896111 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd905b-50bf-4f13-a4b3-7a2f90667358 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.896430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd905b-50bf-4f13-a4b3-7a2f90667358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.897149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd905b-50bf-4f13-a4b3-7a2f90667358 None admin] RBAC: Adding query filter params (role.id=770887c5c808476d8505bf8b8fc5f27a, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=70742ba8f3f6461f8f578b323afea7cc) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:57.897452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd905b-50bf-4f13-a4b3-7a2f90667358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.898881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd905b-50bf-4f13-a4b3-7a2f90667358 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.900229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c779c-1bcf-4639-9eb5-d02b55ce54ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:57.901483 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c779c-1bcf-4639-9eb5-d02b55ce54ef None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:57.902061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c779c-1bcf-4639-9eb5-d02b55ce54ef None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:57.902537 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c779c-1bcf-4639-9eb5-d02b55ce54ef None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:57.903449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c779c-1bcf-4639-9eb5-d02b55ce54ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:57.903904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c779c-1bcf-4639-9eb5-d02b55ce54ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:57.905544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c779c-1bcf-4639-9eb5-d02b55ce54ef None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:57.912412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 246/487] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/role_assignments?user.id=70742ba8f3f6461f8f578b323afea7cc&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=770887c5c808476d8505bf8b8fc5f27a => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:57.912412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd905b-50bf-4f13-a4b3-7a2f90667358 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.920805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 242/488] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:57.927478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c779c-1bcf-4639-9eb5-d02b55ce54ef None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:57.959972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:57.987975 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:58.035992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd905b-50bf-4f13-a4b3-7a2f90667358 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:58.063409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c779c-1bcf-4639-9eb5-d02b55ce54ef None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:58.071929 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f8056-8882-446c-95d2-97d7cb893fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:58.072608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f8056-8882-446c-95d2-97d7cb893fa6 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:58.072883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f8056-8882-446c-95d2-97d7cb893fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:58.074339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f8056-8882-446c-95d2-97d7cb893fa6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:58.085233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f8056-8882-446c-95d2-97d7cb893fa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-675f8056-8882-446c-95d2-97d7cb893fa6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:28:58.090569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:58.091445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:28:58.092231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 247/489] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:58.099567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f8056-8882-446c-95d2-97d7cb893fa6 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:58.122781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7fa21-9319-4fc0-9972-8200d2d58e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:58.125293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7fa21-9319-4fc0-9972-8200d2d58e07 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:58.125943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7fa21-9319-4fc0-9972-8200d2d58e07 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:58.127134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7fa21-9319-4fc0-9972-8200d2d58e07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:58.128290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7fa21-9319-4fc0-9972-8200d2d58e07 None admin] RBAC: Adding query filter params (role.id=770887c5c808476d8505bf8b8fc5f27a, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=ae4b023dbf114bf98d0e38ea7fec94fa) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:58.130434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7fa21-9319-4fc0-9972-8200d2d58e07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:58.132318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7fa21-9319-4fc0-9972-8200d2d58e07 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:58.147351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 243/490] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:28:57 2022] GET /identity/v3/role_assignments?user.id=ae4b023dbf114bf98d0e38ea7fec94fa&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=770887c5c808476d8505bf8b8fc5f27a => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:58.294592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f8056-8882-446c-95d2-97d7cb893fa6 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:58.325200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:58.350904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17544e75-9aa1-4246-9cb2-0721296ba8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:58.351452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17544e75-9aa1-4246-9cb2-0721296ba8f3 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:58.351731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17544e75-9aa1-4246-9cb2-0721296ba8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:58.352004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17544e75-9aa1-4246-9cb2-0721296ba8f3 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:58.352805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17544e75-9aa1-4246-9cb2-0721296ba8f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:58.353068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17544e75-9aa1-4246-9cb2-0721296ba8f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:58.354509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17544e75-9aa1-4246-9cb2-0721296ba8f3 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:58.369423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 248/491] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:28:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:58.375563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7fa21-9319-4fc0-9972-8200d2d58e07 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:58.477603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:58.507287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7fa21-9319-4fc0-9972-8200d2d58e07 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:58.542753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aebf8-cb7b-416c-8cf1-898f4acf9f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:58.543732 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aebf8-cb7b-416c-8cf1-898f4acf9f11 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:58.544213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aebf8-cb7b-416c-8cf1-898f4acf9f11 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:58.544709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aebf8-cb7b-416c-8cf1-898f4acf9f11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:58.555727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2aebf8-cb7b-416c-8cf1-898f4acf9f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:58.558216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2aebf8-cb7b-416c-8cf1-898f4acf9f11 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-de2aebf8-cb7b-416c-8cf1-898f4acf9f11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:58.572946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:58.574351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:58.575690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:58.575690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:58.575690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:58.577862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 244/492] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:58.583632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17544e75-9aa1-4246-9cb2-0721296ba8f3 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:58.703235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17544e75-9aa1-4246-9cb2-0721296ba8f3 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:58.743728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ba10a-e92e-4d05-be4e-e152d33c4390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:58.744737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ba10a-e92e-4d05-be4e-e152d33c4390 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:58.745963 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ba10a-e92e-4d05-be4e-e152d33c4390 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:58.746437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ba10a-e92e-4d05-be4e-e152d33c4390 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:58.747297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ba10a-e92e-4d05-be4e-e152d33c4390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:58.747800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ba10a-e92e-4d05-be4e-e152d33c4390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:58.749431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ba10a-e92e-4d05-be4e-e152d33c4390 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:58.762009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 249/493] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:58.767742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aebf8-cb7b-416c-8cf1-898f4acf9f11 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:58.910499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aebf8-cb7b-416c-8cf1-898f4acf9f11 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:58.958182 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fde63f-47fb-4694-a421-4c880a4d22fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:58.958741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fde63f-47fb-4694-a421-4c880a4d22fb None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:58.959018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fde63f-47fb-4694-a421-4c880a4d22fb None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:58.959288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fde63f-47fb-4694-a421-4c880a4d22fb None admin] PATH_INFO: `/v3/users/swift` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:58.961353 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:58.973860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fde63f-47fb-4694-a421-4c880a4d22fb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:58.975427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fde63f-47fb-4694-a421-4c880a4d22fb None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-f0fde63f-47fb-4694-a421-4c880a4d22fb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:28:58.993528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:58.994496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:58.996047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 245/494] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:28:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:59.003171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ba10a-e92e-4d05-be4e-e152d33c4390 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:59.077814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:59.153779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ba10a-e92e-4d05-be4e-e152d33c4390 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:59.199453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068b22b-eeb4-46cf-9d87-80f5fe6c1c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:59.200056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b22b-eeb4-46cf-9d87-80f5fe6c1c96 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:59.200337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b22b-eeb4-46cf-9d87-80f5fe6c1c96 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:59.200631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b22b-eeb4-46cf-9d87-80f5fe6c1c96 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:59.201356 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068b22b-eeb4-46cf-9d87-80f5fe6c1c96 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:28:59.201659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068b22b-eeb4-46cf-9d87-80f5fe6c1c96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:59.203085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068b22b-eeb4-46cf-9d87-80f5fe6c1c96 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:59.226194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 250/495] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Oct 20 14:28:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:28:59.232066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fde63f-47fb-4694-a421-4c880a4d22fb None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:59.368277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fde63f-47fb-4694-a421-4c880a4d22fb None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:59.414000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6707ec-21bf-4e9c-8fcf-3b63382975b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:59.414636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6707ec-21bf-4e9c-8fcf-3b63382975b6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:59.414915 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6707ec-21bf-4e9c-8fcf-3b63382975b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:59.415187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6707ec-21bf-4e9c-8fcf-3b63382975b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:59.497475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:59.509395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6707ec-21bf-4e9c-8fcf-3b63382975b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:59.511885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6707ec-21bf-4e9c-8fcf-3b63382975b6 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:59.561874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af2fdc-44dd-441b-a312-c72ddc6690a6 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:59.562182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af2fdc-44dd-441b-a312-c72ddc6690a6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:59.562428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af2fdc-44dd-441b-a312-c72ddc6690a6 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:59.563876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 251/496] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:28:59.568877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b050e78-3c15-4cf2-bb7e-93cb6e22a44f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:59.569162 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b050e78-3c15-4cf2-bb7e-93cb6e22a44f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:59.569438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b050e78-3c15-4cf2-bb7e-93cb6e22a44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:59.599489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4b050e78-3c15-4cf2-bb7e-93cb6e22a44f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:28:59.600697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4b050e78-3c15-4cf2-bb7e-93cb6e22a44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:28:59.686187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 246/497] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 14:28:59.690998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6707ec-21bf-4e9c-8fcf-3b63382975b6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:59.711133 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 252/498] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:28:59.833965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:28:59.842045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6707ec-21bf-4e9c-8fcf-3b63382975b6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:28:59.877291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11c5f-afbf-4a65-b08d-3f0bb6f677d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:28:59.878261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11c5f-afbf-4a65-b08d-3f0bb6f677d4 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:28:59.878744 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11c5f-afbf-4a65-b08d-3f0bb6f677d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:28:59.879222 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11c5f-afbf-4a65-b08d-3f0bb6f677d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:28:59.891660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11c5f-afbf-4a65-b08d-3f0bb6f677d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:28:59.893439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11c5f-afbf-4a65-b08d-3f0bb6f677d4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-1ee11c5f-afbf-4a65-b08d-3f0bb6f677d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:28:59.909227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:28:59.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:28:59.910996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:28:59.910996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:28:59.910996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:28:59.913096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 247/499] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:28:59.920035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b050e78-3c15-4cf2-bb7e-93cb6e22a44f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:28:59.972784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:00.006637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11c5f-afbf-4a65-b08d-3f0bb6f677d4 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:00.058032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b050e78-3c15-4cf2-bb7e-93cb6e22a44f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:00.094997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78381773-7935-4024-be4f-297a125bcd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:00.096021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78381773-7935-4024-be4f-297a125bcd36 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.096562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78381773-7935-4024-be4f-297a125bcd36 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.096850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78381773-7935-4024-be4f-297a125bcd36 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.097496 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78381773-7935-4024-be4f-297a125bcd36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:00.097810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78381773-7935-4024-be4f-297a125bcd36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:00.104372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78381773-7935-4024-be4f-297a125bcd36 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:00.116385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 253/500] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:00.119281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78381773-7935-4024-be4f-297a125bcd36 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:00.234894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78381773-7935-4024-be4f-297a125bcd36 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:00.250144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:00.255775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11c5f-afbf-4a65-b08d-3f0bb6f677d4 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:00.271583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b6a19-d8a1-4e1a-9886-04776c9d30e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:00.272226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b6a19-d8a1-4e1a-9886-04776c9d30e3 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.272561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b6a19-d8a1-4e1a-9886-04776c9d30e3 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.272841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b6a19-d8a1-4e1a-9886-04776c9d30e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.280265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b6a19-d8a1-4e1a-9886-04776c9d30e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:00.281769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b6a19-d8a1-4e1a-9886-04776c9d30e3 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-b05b6a19-d8a1-4e1a-9886-04776c9d30e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:00.295535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:00.296536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:00.297865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:00.297865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:00.297865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:00.297865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:00.297865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:00.297865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:00.297865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:00.297865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:00.301523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 254/501] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:00.302728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b6a19-d8a1-4e1a-9886-04776c9d30e3 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:00.328540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03595a5-1e5f-42bb-bf59-ae040e19defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:00.329515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03595a5-1e5f-42bb-bf59-ae040e19defd None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.329999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03595a5-1e5f-42bb-bf59-ae040e19defd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.330477 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03595a5-1e5f-42bb-bf59-ae040e19defd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.357291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03595a5-1e5f-42bb-bf59-ae040e19defd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-c03595a5-1e5f-42bb-bf59-ae040e19defd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:00.364965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:00.365949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:00.366749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:00.374042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 248/502] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:29:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:00.379973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03595a5-1e5f-42bb-bf59-ae040e19defd None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:00.396517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:00.435288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b6a19-d8a1-4e1a-9886-04776c9d30e3 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:00.509016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325841e9-0eb2-4f34-9635-e3113db9c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:00.510037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325841e9-0eb2-4f34-9635-e3113db9c67d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.510741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325841e9-0eb2-4f34-9635-e3113db9c67d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.511372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325841e9-0eb2-4f34-9635-e3113db9c67d None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.512301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325841e9-0eb2-4f34-9635-e3113db9c67d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:00.513080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325841e9-0eb2-4f34-9635-e3113db9c67d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:00.516324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325841e9-0eb2-4f34-9635-e3113db9c67d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:00.532643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 255/503] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:00.538396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325841e9-0eb2-4f34-9635-e3113db9c67d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:00.572285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:00.581199 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03595a5-1e5f-42bb-bf59-ae040e19defd None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:00.669443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325841e9-0eb2-4f34-9635-e3113db9c67d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:00.680645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37694c4a-de3b-4e62-b8a5-ed99eaf0e884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:00.681262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37694c4a-de3b-4e62-b8a5-ed99eaf0e884 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.681544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37694c4a-de3b-4e62-b8a5-ed99eaf0e884 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.681819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37694c4a-de3b-4e62-b8a5-ed99eaf0e884 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.683008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37694c4a-de3b-4e62-b8a5-ed99eaf0e884 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:00.683316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37694c4a-de3b-4e62-b8a5-ed99eaf0e884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:00.684794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37694c4a-de3b-4e62-b8a5-ed99eaf0e884 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:00.693045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 249/504] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:29:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:00.694265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8efdbc-488a-4c83-9724-a004c7351264 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.694564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8efdbc-488a-4c83-9724-a004c7351264 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.694844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8efdbc-488a-4c83-9724-a004c7351264 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.700291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 250/505] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:00.701487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdcd91-f08f-4332-9b25-ee43cadff5bd None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.701880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdcd91-f08f-4332-9b25-ee43cadff5bd None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.702119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdcd91-f08f-4332-9b25-ee43cadff5bd None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.703279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 251/506] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:00.704542 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdcd91-f08f-4332-9b25-ee43cadff5bd None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:00.713157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14738bf6-ccac-4bc1-b44e-a8329ac05c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:00.714289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14738bf6-ccac-4bc1-b44e-a8329ac05c37 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.714632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14738bf6-ccac-4bc1-b44e-a8329ac05c37 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.715344 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14738bf6-ccac-4bc1-b44e-a8329ac05c37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.716104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14738bf6-ccac-4bc1-b44e-a8329ac05c37 None admin] RBAC: Adding query filter params (role.id=770887c5c808476d8505bf8b8fc5f27a, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=61d6c1cd010d4fa387fa64864419bc6a) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:00.716452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14738bf6-ccac-4bc1-b44e-a8329ac05c37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:00.718566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14738bf6-ccac-4bc1-b44e-a8329ac05c37 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:00.732374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 256/507] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:29:00 2022] GET /identity/v3/role_assignments?user.id=61d6c1cd010d4fa387fa64864419bc6a&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=770887c5c808476d8505bf8b8fc5f27a => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:00.732374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f7aaa-4964-4efd-9a5a-0a2267d516f0 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.732374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f7aaa-4964-4efd-9a5a-0a2267d516f0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.732374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f7aaa-4964-4efd-9a5a-0a2267d516f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.749229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e82f7aaa-4964-4efd-9a5a-0a2267d516f0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:00.750523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e82f7aaa-4964-4efd-9a5a-0a2267d516f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:00.794906 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:00.836527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdcd91-f08f-4332-9b25-ee43cadff5bd None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:00.842661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 257/508] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:29:00.844062 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8c4e7-02fe-4ef5-84e5-f3210080d0cb None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.844368 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8c4e7-02fe-4ef5-84e5-f3210080d0cb None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.844655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8c4e7-02fe-4ef5-84e5-f3210080d0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.851996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:00.873109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-99a8c4e7-02fe-4ef5-84e5-f3210080d0cb None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:00.874388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-99a8c4e7-02fe-4ef5-84e5-f3210080d0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:00.922594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1054e7-481b-4998-86dd-9bfb3b24cc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:00.923575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1054e7-481b-4998-86dd-9bfb3b24cc1d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.924076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1054e7-481b-4998-86dd-9bfb3b24cc1d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.924592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1054e7-481b-4998-86dd-9bfb3b24cc1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:00.937005 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1054e7-481b-4998-86dd-9bfb3b24cc1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:00.938754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1054e7-481b-4998-86dd-9bfb3b24cc1d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-ed1054e7-481b-4998-86dd-9bfb3b24cc1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:00.959032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:00.960008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:00.960912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:00.960912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:00.960912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:00.966967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 252/509] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:00.968913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1054e7-481b-4998-86dd-9bfb3b24cc1d None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:00.989224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 258/510] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:29:00.999313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f230b-9f28-4d23-b7d0-05f9c93080af None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:00.999591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f230b-9f28-4d23-b7d0-05f9c93080af None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:00.999867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f230b-9f28-4d23-b7d0-05f9c93080af None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.001966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 259/511] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:29:01.006588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64d690-3880-4f25-81da-4c780898053a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.007090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64d690-3880-4f25-81da-4c780898053a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.007560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64d690-3880-4f25-81da-4c780898053a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.030274 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-5e64d690-3880-4f25-81da-4c780898053a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:01.031939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5e64d690-3880-4f25-81da-4c780898053a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:01.085152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1054e7-481b-4998-86dd-9bfb3b24cc1d None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:01.106818 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:01.118907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 260/512] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:29:01.137119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc253a8-5a13-48f1-85ca-90ed485265a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:01.137785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc253a8-5a13-48f1-85ca-90ed485265a9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.138066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc253a8-5a13-48f1-85ca-90ed485265a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.138338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc253a8-5a13-48f1-85ca-90ed485265a9 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.138987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc253a8-5a13-48f1-85ca-90ed485265a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:01.139289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc253a8-5a13-48f1-85ca-90ed485265a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:01.140756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc253a8-5a13-48f1-85ca-90ed485265a9 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:01.144133 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64d690-3880-4f25-81da-4c780898053a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.156374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 253/513] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:01.163513 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc253a8-5a13-48f1-85ca-90ed485265a9 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.266116 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:01.316719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64d690-3880-4f25-81da-4c780898053a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:01.320985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc253a8-5a13-48f1-85ca-90ed485265a9 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:01.359808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbafc8b-d35d-4e2e-9d00-7f6775f5461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:01.360366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbafc8b-d35d-4e2e-9d00-7f6775f5461a None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.360649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbafc8b-d35d-4e2e-9d00-7f6775f5461a None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.360924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbafc8b-d35d-4e2e-9d00-7f6775f5461a None admin] PATH_INFO: `/v3/users/nova` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.365479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7e4cd-bfd9-4c5c-b121-1e29221b5664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:01.366336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7e4cd-bfd9-4c5c-b121-1e29221b5664 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.366807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7e4cd-bfd9-4c5c-b121-1e29221b5664 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.367266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7e4cd-bfd9-4c5c-b121-1e29221b5664 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.368299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7e4cd-bfd9-4c5c-b121-1e29221b5664 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:01.368549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbafc8b-d35d-4e2e-9d00-7f6775f5461a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:01.370540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbafc8b-d35d-4e2e-9d00-7f6775f5461a None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:01.371226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7e4cd-bfd9-4c5c-b121-1e29221b5664 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-2cbafc8b-d35d-4e2e-9d00-7f6775f5461a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:01.382246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:01.383168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 254/514] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:01.384062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbafc8b-d35d-4e2e-9d00-7f6775f5461a None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-61f7e4cd-bfd9-4c5c-b121-1e29221b5664 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:01.385866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:01.386786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 14:29:01.387586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:01.390211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 261/515] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:01.396155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7e4cd-bfd9-4c5c-b121-1e29221b5664 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.431725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:01.501776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbafc8b-d35d-4e2e-9d00-7f6775f5461a None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:01.507647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7e4cd-bfd9-4c5c-b121-1e29221b5664 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:01.535044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:01.544405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8ad57-adb6-49ec-be54-3dbe80364ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:01.545021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8ad57-adb6-49ec-be54-3dbe80364ccf None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.545285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8ad57-adb6-49ec-be54-3dbe80364ccf None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.545562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8ad57-adb6-49ec-be54-3dbe80364ccf None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.546176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8ad57-adb6-49ec-be54-3dbe80364ccf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:01.546474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8ad57-adb6-49ec-be54-3dbe80364ccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:01.547274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aed25-7e84-4435-a0ab-2ef8c0ef0145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:01.547850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8ad57-adb6-49ec-be54-3dbe80364ccf None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:01.548206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aed25-7e84-4435-a0ab-2ef8c0ef0145 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.548648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aed25-7e84-4435-a0ab-2ef8c0ef0145 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.549225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aed25-7e84-4435-a0ab-2ef8c0ef0145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.552392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 262/516] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:01.553459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8ad57-adb6-49ec-be54-3dbe80364ccf None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.561543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1aed25-7e84-4435-a0ab-2ef8c0ef0145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:01.563003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1aed25-7e84-4435-a0ab-2ef8c0ef0145 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-7a1aed25-7e84-4435-a0ab-2ef8c0ef0145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:01.577279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:01.578218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:01.579114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:01.579114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:01.579114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:01.579114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 255/517] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:01.579114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aed25-7e84-4435-a0ab-2ef8c0ef0145 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.666009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8ad57-adb6-49ec-be54-3dbe80364ccf None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:01.674782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:01.685238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aed25-7e84-4435-a0ab-2ef8c0ef0145 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:01.700921 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82822ae5-1b9a-4100-bf5f-7e7f0fde2a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:01.701528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82822ae5-1b9a-4100-bf5f-7e7f0fde2a78 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.701806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82822ae5-1b9a-4100-bf5f-7e7f0fde2a78 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.702074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82822ae5-1b9a-4100-bf5f-7e7f0fde2a78 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.702847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82822ae5-1b9a-4100-bf5f-7e7f0fde2a78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:01.703156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82822ae5-1b9a-4100-bf5f-7e7f0fde2a78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:01.704616 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82822ae5-1b9a-4100-bf5f-7e7f0fde2a78 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:01.722067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5b94-0389-4e2f-8d47-542bf6b50268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:01.722719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5b94-0389-4e2f-8d47-542bf6b50268 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.727593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5b94-0389-4e2f-8d47-542bf6b50268 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.727708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 263/518] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:01.728067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5b94-0389-4e2f-8d47-542bf6b50268 None admin] PATH_INFO: `/v3/services/network` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.728140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82822ae5-1b9a-4100-bf5f-7e7f0fde2a78 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.728209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca5b94-0389-4e2f-8d47-542bf6b50268 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:01.728209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca5b94-0389-4e2f-8d47-542bf6b50268 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-beca5b94-0389-4e2f-8d47-542bf6b50268 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:01.739683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:01.740757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:01.741565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 256/519] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:01.742574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5b94-0389-4e2f-8d47-542bf6b50268 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.785190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:01.839846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82822ae5-1b9a-4100-bf5f-7e7f0fde2a78 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:01.848601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5b94-0389-4e2f-8d47-542bf6b50268 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:01.879839 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546153d-2d62-438b-8e55-205ef37fb56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:01.880607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546153d-2d62-438b-8e55-205ef37fb56a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.880889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546153d-2d62-438b-8e55-205ef37fb56a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.881394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546153d-2d62-438b-8e55-205ef37fb56a None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.881800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546153d-2d62-438b-8e55-205ef37fb56a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:01.882108 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546153d-2d62-438b-8e55-205ef37fb56a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:01.883487 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546153d-2d62-438b-8e55-205ef37fb56a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:01.886746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2fd0-0fb5-4476-a530-7584936731af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:01.887356 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2fd0-0fb5-4476-a530-7584936731af None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:01.887633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2fd0-0fb5-4476-a530-7584936731af None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:01.887902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2fd0-0fb5-4476-a530-7584936731af None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:01.888579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da2fd0-0fb5-4476-a530-7584936731af None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:01.888870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da2fd0-0fb5-4476-a530-7584936731af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:01.890207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 264/520] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:01.890327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da2fd0-0fb5-4476-a530-7584936731af None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:01.893849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546153d-2d62-438b-8e55-205ef37fb56a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.909286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 257/521] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:01.909978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2fd0-0fb5-4476-a530-7584936731af None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:01.930549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:02.032816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2fd0-0fb5-4476-a530-7584936731af None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:02.040967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546153d-2d62-438b-8e55-205ef37fb56a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:02.058077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:02.089666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea260a-86c9-4d14-bb33-cddf0aa0bb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:02.090311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea260a-86c9-4d14-bb33-cddf0aa0bb68 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:02.090592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea260a-86c9-4d14-bb33-cddf0aa0bb68 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:02.090868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea260a-86c9-4d14-bb33-cddf0aa0bb68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:02.091811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1a250-06d0-4768-8926-4f4cc15d6ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:02.092660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1a250-06d0-4768-8926-4f4cc15d6ab1 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:02.093134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1a250-06d0-4768-8926-4f4cc15d6ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:02.093593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1a250-06d0-4768-8926-4f4cc15d6ab1 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:02.094226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1a250-06d0-4768-8926-4f4cc15d6ab1 None admin] RBAC: Adding query filter params (name=network) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:02.094529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1a250-06d0-4768-8926-4f4cc15d6ab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:02.095901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1a250-06d0-4768-8926-4f4cc15d6ab1 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:02.101117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 258/522] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:02.101956 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1a250-06d0-4768-8926-4f4cc15d6ab1 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:02.110810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea260a-86c9-4d14-bb33-cddf0aa0bb68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:02.112304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea260a-86c9-4d14-bb33-cddf0aa0bb68 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-7aea260a-86c9-4d14-bb33-cddf0aa0bb68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:02.133584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:02.134553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:02.135502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:02.135502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:02.135502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:02.135681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 265/523] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:02.136840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea260a-86c9-4d14-bb33-cddf0aa0bb68 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:02.197250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:02.217436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1a250-06d0-4768-8926-4f4cc15d6ab1 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:02.254177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7eef3-495a-4ebb-b632-41a194d2b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:02.254969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7eef3-495a-4ebb-b632-41a194d2b2be None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:02.255497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7eef3-495a-4ebb-b632-41a194d2b2be None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:02.255867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7eef3-495a-4ebb-b632-41a194d2b2be None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:02.256737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7eef3-495a-4ebb-b632-41a194d2b2be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:02.258364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7eef3-495a-4ebb-b632-41a194d2b2be None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:02.273453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea260a-86c9-4d14-bb33-cddf0aa0bb68 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:02.283566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 259/524] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:29:02 2022] POST /identity/v3/users => generated 254 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:02.284514 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7eef3-495a-4ebb-b632-41a194d2b2be None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:02.345865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965422d-8a39-4679-bc47-002c1c13dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:02.346406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965422d-8a39-4679-bc47-002c1c13dc2d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:02.346685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965422d-8a39-4679-bc47-002c1c13dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:02.346957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965422d-8a39-4679-bc47-002c1c13dc2d None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:02.347593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965422d-8a39-4679-bc47-002c1c13dc2d None admin] RBAC: Adding query filter params (type=network) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:02.347901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965422d-8a39-4679-bc47-002c1c13dc2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:02.349312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965422d-8a39-4679-bc47-002c1c13dc2d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:02.364548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 266/525] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:02.444737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7eef3-495a-4ebb-b632-41a194d2b2be None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:02.501965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa5e38-695d-4320-93bd-a4d268112bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:02.502946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5e38-695d-4320-93bd-a4d268112bdf None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:02.503365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5e38-695d-4320-93bd-a4d268112bdf None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:02.503774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5e38-695d-4320-93bd-a4d268112bdf None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:02.504591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa5e38-695d-4320-93bd-a4d268112bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:02.505128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa5e38-695d-4320-93bd-a4d268112bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:02.506672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa5e38-695d-4320-93bd-a4d268112bdf None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:02.511022 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:02.526328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 260/526] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:02.532650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965422d-8a39-4679-bc47-002c1c13dc2d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:02.572379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:02.649941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965422d-8a39-4679-bc47-002c1c13dc2d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:02.690960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878154b-9ad1-4f2e-80d5-332ba70558ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:02.691659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878154b-9ad1-4f2e-80d5-332ba70558ed None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:02.691889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878154b-9ad1-4f2e-80d5-332ba70558ed None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:02.692167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878154b-9ad1-4f2e-80d5-332ba70558ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:02.702443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878154b-9ad1-4f2e-80d5-332ba70558ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:02.703973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878154b-9ad1-4f2e-80d5-332ba70558ed None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-d878154b-9ad1-4f2e-80d5-332ba70558ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:02.718221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:02.719130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:02.720002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:02.720002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:02.720002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:02.720002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:02.720002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:02.720002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:02.720002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:02.720002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:02.720002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 267/527] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:02.725213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa5e38-695d-4320-93bd-a4d268112bdf None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:02.856549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa5e38-695d-4320-93bd-a4d268112bdf None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:02.911578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdb1bf-acb7-4da8-b3ff-8756977836d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:02.912113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdb1bf-acb7-4da8-b3ff-8756977836d7 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:02.912414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdb1bf-acb7-4da8-b3ff-8756977836d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:02.912690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdb1bf-acb7-4da8-b3ff-8756977836d7 None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:02.913380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdb1bf-acb7-4da8-b3ff-8756977836d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:02.913684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdb1bf-acb7-4da8-b3ff-8756977836d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:02.915133 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdb1bf-acb7-4da8-b3ff-8756977836d7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:02.936440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 261/528] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:02.942154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878154b-9ad1-4f2e-80d5-332ba70558ed None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:03.069185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:03.080607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878154b-9ad1-4f2e-80d5-332ba70558ed None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:03.124768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bba236-fc7a-4e41-9d0d-f2ee416d85f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:03.125434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bba236-fc7a-4e41-9d0d-f2ee416d85f4 None admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:03.125712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bba236-fc7a-4e41-9d0d-f2ee416d85f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:03.125985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bba236-fc7a-4e41-9d0d-f2ee416d85f4 None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/9647f8caa6cd4208800e2371fdf28301/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:03.127903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bba236-fc7a-4e41-9d0d-f2ee416d85f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=9647f8caa6cd4208800e2371fdf28301, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:03.129377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bba236-fc7a-4e41-9d0d-f2ee416d85f4 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:03.144762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 268/529] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:29:02 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/9647f8caa6cd4208800e2371fdf28301/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:29:03.328791 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46757915-347b-4480-8a96-c2adb265d37a None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:03.329175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46757915-347b-4480-8a96-c2adb265d37a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:03.329352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46757915-347b-4480-8a96-c2adb265d37a None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:03.330943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 262/530] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:03.336535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a6778-f765-41fe-b35e-738b97b475ab None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:03.336817 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a6778-f765-41fe-b35e-738b97b475ab None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:03.337104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a6778-f765-41fe-b35e-738b97b475ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:03.362405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-010a6778-f765-41fe-b35e-738b97b475ab None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:03.363642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-010a6778-f765-41fe-b35e-738b97b475ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:03.467079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 269/531] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:29:03.833960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46757915-347b-4480-8a96-c2adb265d37a None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:03.910501 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:03.979027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46757915-347b-4480-8a96-c2adb265d37a None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:04.037935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c631b-239f-4013-abaf-9e550a6325f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:04.038596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c631b-239f-4013-abaf-9e550a6325f3 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:04.038887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c631b-239f-4013-abaf-9e550a6325f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:04.039171 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c631b-239f-4013-abaf-9e550a6325f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:04.054715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c631b-239f-4013-abaf-9e550a6325f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-072c631b-239f-4013-abaf-9e550a6325f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:04.059705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:04.060612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:04.061424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 263/532] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:29:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:04.066925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a6778-f765-41fe-b35e-738b97b475ab None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:04.211509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:04.237564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a6778-f765-41fe-b35e-738b97b475ab None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:04.273537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796afb7e-bc54-4e02-ada5-58d996aa78b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:04.274137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796afb7e-bc54-4e02-ada5-58d996aa78b5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:04.274416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796afb7e-bc54-4e02-ada5-58d996aa78b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:04.274688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796afb7e-bc54-4e02-ada5-58d996aa78b5 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:04.275352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796afb7e-bc54-4e02-ada5-58d996aa78b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:04.275656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796afb7e-bc54-4e02-ada5-58d996aa78b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:04.277718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796afb7e-bc54-4e02-ada5-58d996aa78b5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:04.286774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 270/533] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:04.292123 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c631b-239f-4013-abaf-9e550a6325f3 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:04.439167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1135e-a9f7-4a9b-a754-509d4c6f9a07 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:04.439846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1135e-a9f7-4a9b-a754-509d4c6f9a07 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:04.440317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1135e-a9f7-4a9b-a754-509d4c6f9a07 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:04.442110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 271/534] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:04.447331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49bd48-1f3a-46e4-bf55-c6d1132575aa None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:04.448145 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49bd48-1f3a-46e4-bf55-c6d1132575aa None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:04.448667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49bd48-1f3a-46e4-bf55-c6d1132575aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:04.458252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c631b-239f-4013-abaf-9e550a6325f3 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:04.469236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-dc49bd48-1f3a-46e4-bf55-c6d1132575aa None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:04.470753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-dc49bd48-1f3a-46e4-bf55-c6d1132575aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:04.493509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:04.521603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb4f1a-4c38-41bb-ad28-5d6ad42a4f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:04.522545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb4f1a-4c38-41bb-ad28-5d6ad42a4f6e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:04.528455 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb4f1a-4c38-41bb-ad28-5d6ad42a4f6e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:04.528945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb4f1a-4c38-41bb-ad28-5d6ad42a4f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:04.547080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb4f1a-4c38-41bb-ad28-5d6ad42a4f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:04.548574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb4f1a-4c38-41bb-ad28-5d6ad42a4f6e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:04.567775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 272/535] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-94bb4f1a-4c38-41bb-ad28-5d6ad42a4f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:04.582007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:04.583015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:04.583869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:04.583869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:04.583869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:04.587905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 264/536] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:04.589960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49bd48-1f3a-46e4-bf55-c6d1132575aa None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:04.687154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:04.717809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49bd48-1f3a-46e4-bf55-c6d1132575aa None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:04.765362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3f45e-f48f-4286-8b66-c07a522b682d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:04.765895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3f45e-f48f-4286-8b66-c07a522b682d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:04.766235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3f45e-f48f-4286-8b66-c07a522b682d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:04.766519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3f45e-f48f-4286-8b66-c07a522b682d None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:04.767166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3f45e-f48f-4286-8b66-c07a522b682d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:04.767473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3f45e-f48f-4286-8b66-c07a522b682d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:04.768884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3f45e-f48f-4286-8b66-c07a522b682d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:04.789300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 273/537] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:04.797024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb4f1a-4c38-41bb-ad28-5d6ad42a4f6e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:04.946100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb4f1a-4c38-41bb-ad28-5d6ad42a4f6e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:04.979472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3f45e-f48f-4286-8b66-c07a522b682d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:04.991755 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dbfa9-a508-4a06-a6a7-100045fa0424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:04.992665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dbfa9-a508-4a06-a6a7-100045fa0424 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:04.992947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dbfa9-a508-4a06-a6a7-100045fa0424 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:04.993219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dbfa9-a508-4a06-a6a7-100045fa0424 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:05.009447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dbfa9-a508-4a06-a6a7-100045fa0424 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:05.011330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dbfa9-a508-4a06-a6a7-100045fa0424 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-2b1dbfa9-a508-4a06-a6a7-100045fa0424 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:05.025295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:05.026749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 14:29:05.027774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:05.030232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 265/538] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:29:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:05.035892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dbfa9-a508-4a06-a6a7-100045fa0424 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:05.089319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:05.125097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3f45e-f48f-4286-8b66-c07a522b682d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:05.162543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d4912-4b39-47e2-9d41-7b0275290700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:05.163163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d4912-4b39-47e2-9d41-7b0275290700 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:05.163501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d4912-4b39-47e2-9d41-7b0275290700 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:05.163793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d4912-4b39-47e2-9d41-7b0275290700 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:05.164494 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d4912-4b39-47e2-9d41-7b0275290700 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:05.166140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d4912-4b39-47e2-9d41-7b0275290700 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:05.172290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:05.197491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 274/539] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:29:04 2022] POST /identity/v3/services => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:29:05.237533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dbfa9-a508-4a06-a6a7-100045fa0424 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:05.302933 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e27151-acab-4e97-9c2e-f0201cac88de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:05.303541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e27151-acab-4e97-9c2e-f0201cac88de None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:05.303818 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e27151-acab-4e97-9c2e-f0201cac88de None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:05.304090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e27151-acab-4e97-9c2e-f0201cac88de None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:05.304858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e27151-acab-4e97-9c2e-f0201cac88de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:05.305164 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e27151-acab-4e97-9c2e-f0201cac88de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:05.306646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e27151-acab-4e97-9c2e-f0201cac88de None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:05.329277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 266/540] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Oct 20 14:29:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:05.334971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d4912-4b39-47e2-9d41-7b0275290700 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:05.438623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a20584-d6c6-44c5-8a4a-7435c3788f24 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:05.438958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a20584-d6c6-44c5-8a4a-7435c3788f24 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:05.439181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a20584-d6c6-44c5-8a4a-7435c3788f24 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:05.440805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 267/541] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:05.449872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f2cb4-5e58-4b92-9d33-5f70d9408711 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:05.450210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f2cb4-5e58-4b92-9d33-5f70d9408711 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:05.450420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f2cb4-5e58-4b92-9d33-5f70d9408711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:05.474814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-792f2cb4-5e58-4b92-9d33-5f70d9408711 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:05.476001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-792f2cb4-5e58-4b92-9d33-5f70d9408711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:05.483482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d4912-4b39-47e2-9d41-7b0275290700 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:05.519131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b192ddd-a70e-4a05-acff-d22ea787406e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:05.520005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b192ddd-a70e-4a05-acff-d22ea787406e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:05.520447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b192ddd-a70e-4a05-acff-d22ea787406e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:05.520860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b192ddd-a70e-4a05-acff-d22ea787406e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:05.532237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b192ddd-a70e-4a05-acff-d22ea787406e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:05.533921 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b192ddd-a70e-4a05-acff-d22ea787406e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-3b192ddd-a70e-4a05-acff-d22ea787406e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:05.549961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:05.551104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:05.552204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:05.552204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:05.552204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:05.553768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 275/542] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:05.558235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 268/543] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 14:29:05.558378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005fe8-01ae-47a8-b06f-e8672bc66301 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:05.558378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005fe8-01ae-47a8-b06f-e8672bc66301 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:05.558378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005fe8-01ae-47a8-b06f-e8672bc66301 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:05.558378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 276/544] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:05.562099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f2cb4-5e58-4b92-9d33-5f70d9408711 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:05.562610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d578d-05c7-4dc3-9cbc-98126d06e869 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:05.562895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d578d-05c7-4dc3-9cbc-98126d06e869 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:05.563175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d578d-05c7-4dc3-9cbc-98126d06e869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:05.578652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:05.588819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c27d578d-05c7-4dc3-9cbc-98126d06e869 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:05.590653 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c27d578d-05c7-4dc3-9cbc-98126d06e869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:05.644162 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:05.677679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 277/545] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 14:29:05.718115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f2cb4-5e58-4b92-9d33-5f70d9408711 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:05.773158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0580f2d-6519-4afb-a451-71e2c3d34078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:05.773763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0580f2d-6519-4afb-a451-71e2c3d34078 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:05.774039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0580f2d-6519-4afb-a451-71e2c3d34078 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:05.774307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0580f2d-6519-4afb-a451-71e2c3d34078 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:05.774949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0580f2d-6519-4afb-a451-71e2c3d34078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:05.775250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0580f2d-6519-4afb-a451-71e2c3d34078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:05.777180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0580f2d-6519-4afb-a451-71e2c3d34078 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:05.798768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 269/546] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:05.804300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d578d-05c7-4dc3-9cbc-98126d06e869 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:05.953026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0580f2d-6519-4afb-a451-71e2c3d34078 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:05.957172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d578d-05c7-4dc3-9cbc-98126d06e869 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:06.001371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a9991-55d3-4676-8621-763e72a6711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:06.002401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a9991-55d3-4676-8621-763e72a6711a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.002691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a9991-55d3-4676-8621-763e72a6711a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.002967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a9991-55d3-4676-8621-763e72a6711a None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.011185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a9991-55d3-4676-8621-763e72a6711a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:06.012725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a9991-55d3-4676-8621-763e72a6711a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-ed0a9991-55d3-4676-8621-763e72a6711a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:06.026384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:06.027521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:06.028645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:06.028645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:06.028645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:06.028645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:06.028645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:06.028645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:06.028645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:06.028645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:06.030928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 278/547] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:06.037572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a9991-55d3-4676-8621-763e72a6711a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:06.074407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:06.091283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:06.125842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0580f2d-6519-4afb-a451-71e2c3d34078 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:06.146517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a9991-55d3-4676-8621-763e72a6711a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:06.163685 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750a847-a704-4efd-b15c-381f0dd7ebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:06.164399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750a847-a704-4efd-b15c-381f0dd7ebc6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.164627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750a847-a704-4efd-b15c-381f0dd7ebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.165074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750a847-a704-4efd-b15c-381f0dd7ebc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.175975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750a847-a704-4efd-b15c-381f0dd7ebc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-e750a847-a704-4efd-b15c-381f0dd7ebc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:06.181693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:06.182808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:06.183627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 270/548] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:29:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:06.185072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750a847-a704-4efd-b15c-381f0dd7ebc6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:06.189647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee66e7-3630-4182-b10d-54584a06f1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:06.190260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee66e7-3630-4182-b10d-54584a06f1b7 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.190535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee66e7-3630-4182-b10d-54584a06f1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.190803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee66e7-3630-4182-b10d-54584a06f1b7 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.191453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee66e7-3630-4182-b10d-54584a06f1b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:06.192861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee66e7-3630-4182-b10d-54584a06f1b7 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:06.209160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 279/549] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:29:06 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:06.209160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db995a4c-4db1-4724-aa0d-6df8d36a9c79 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.209160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db995a4c-4db1-4724-aa0d-6df8d36a9c79 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.209160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db995a4c-4db1-4724-aa0d-6df8d36a9c79 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.209160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 280/550] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:06.209160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db995a4c-4db1-4724-aa0d-6df8d36a9c79 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:06.330870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:06.331868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750a847-a704-4efd-b15c-381f0dd7ebc6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:06.368457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db995a4c-4db1-4724-aa0d-6df8d36a9c79 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:06.375225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:06.383695 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ada8-e0e2-471f-a7a1-aae998a82ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:06.384540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ada8-e0e2-471f-a7a1-aae998a82ae8 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.384923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ada8-e0e2-471f-a7a1-aae998a82ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.385391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ada8-e0e2-471f-a7a1-aae998a82ae8 None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.386219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ada8-e0e2-471f-a7a1-aae998a82ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:06.386932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ada8-e0e2-471f-a7a1-aae998a82ae8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:06.389162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ada8-e0e2-471f-a7a1-aae998a82ae8 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:06.407444 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 271/551] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:06.409763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fe65c-6f4e-4a31-bc67-1d1cc7941393 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.410055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fe65c-6f4e-4a31-bc67-1d1cc7941393 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.411072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fe65c-6f4e-4a31-bc67-1d1cc7941393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.413946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a3680-62fc-4fa4-b944-cf7ec0e7e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:06.414990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a3680-62fc-4fa4-b944-cf7ec0e7e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.415476 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a3680-62fc-4fa4-b944-cf7ec0e7e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.415972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a3680-62fc-4fa4-b944-cf7ec0e7e3f5 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.418947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a3680-62fc-4fa4-b944-cf7ec0e7e3f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:06.419470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a3680-62fc-4fa4-b944-cf7ec0e7e3f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:06.421135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a3680-62fc-4fa4-b944-cf7ec0e7e3f5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:06.430889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 281/552] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:29:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:06.432779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a3680-62fc-4fa4-b944-cf7ec0e7e3f5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:06.433515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-c08fe65c-6f4e-4a31-bc67-1d1cc7941393 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:06.434836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c08fe65c-6f4e-4a31-bc67-1d1cc7941393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:06.519034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 272/553] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:06.519890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fe65c-6f4e-4a31-bc67-1d1cc7941393 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:06.552929 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a3680-62fc-4fa4-b944-cf7ec0e7e3f5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:06.592477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac514d3-1b3c-4b83-b5c0-3cabca01806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:06.593097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac514d3-1b3c-4b83-b5c0-3cabca01806c None admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.593381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac514d3-1b3c-4b83-b5c0-3cabca01806c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.593657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac514d3-1b3c-4b83-b5c0-3cabca01806c None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/61d6c1cd010d4fa387fa64864419bc6a/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.595626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac514d3-1b3c-4b83-b5c0-3cabca01806c None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=61d6c1cd010d4fa387fa64864419bc6a, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:06.597399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac514d3-1b3c-4b83-b5c0-3cabca01806c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:06.612775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 282/554] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:29:06 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/61d6c1cd010d4fa387fa64864419bc6a/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:29:06.698896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fe65c-6f4e-4a31-bc67-1d1cc7941393 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:06.735170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:06.753169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6c112-eb59-4ef1-849f-97f29a67645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:06.754116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6c112-eb59-4ef1-849f-97f29a67645e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.754538 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6c112-eb59-4ef1-849f-97f29a67645e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.755014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6c112-eb59-4ef1-849f-97f29a67645e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.768395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6c112-eb59-4ef1-849f-97f29a67645e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:06.770174 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6c112-eb59-4ef1-849f-97f29a67645e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-28b6c112-eb59-4ef1-849f-97f29a67645e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:06.787980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:06.789251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:06.790320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:06.790320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:06.790320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:06.792286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 273/555] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:06.800394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac514d3-1b3c-4b83-b5c0-3cabca01806c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:06.820492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:06.913712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac514d3-1b3c-4b83-b5c0-3cabca01806c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:06.949114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b12f8a-7f17-4446-a54b-4c6810bf5e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:06.950126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b12f8a-7f17-4446-a54b-4c6810bf5e08 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:06.950613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b12f8a-7f17-4446-a54b-4c6810bf5e08 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:06.951115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b12f8a-7f17-4446-a54b-4c6810bf5e08 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:06.951983 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b12f8a-7f17-4446-a54b-4c6810bf5e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:06.952593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b12f8a-7f17-4446-a54b-4c6810bf5e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:06.954221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b12f8a-7f17-4446-a54b-4c6810bf5e08 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:06.967209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 283/556] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:06.972262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6c112-eb59-4ef1-849f-97f29a67645e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:06.979621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b12f8a-7f17-4446-a54b-4c6810bf5e08 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:07.107020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:07.142581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b12f8a-7f17-4446-a54b-4c6810bf5e08 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:07.147742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6c112-eb59-4ef1-849f-97f29a67645e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:07.184323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f01594-7337-4600-a479-72814177b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:07.184933 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f01594-7337-4600-a479-72814177b589 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:07.185215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f01594-7337-4600-a479-72814177b589 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:07.185595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f01594-7337-4600-a479-72814177b589 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:07.186303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2e20d-8ae3-4d4e-8f64-92cf7a4863de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:07.187738 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2e20d-8ae3-4d4e-8f64-92cf7a4863de None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:07.188881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2e20d-8ae3-4d4e-8f64-92cf7a4863de None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:07.189373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2e20d-8ae3-4d4e-8f64-92cf7a4863de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:07.202403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f01594-7337-4600-a479-72814177b589 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:07.205256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f01594-7337-4600-a479-72814177b589 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:07.213399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2e20d-8ae3-4d4e-8f64-92cf7a4863de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-70b2e20d-8ae3-4d4e-8f64-92cf7a4863de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:07.222067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:07.222961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:07.223800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:07.226353 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 284/557] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:29:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-04f01594-7337-4600-a479-72814177b589 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:07.231797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:07.232733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 14:29:07.233617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:07.235387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2e20d-8ae3-4d4e-8f64-92cf7a4863de None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:07.237106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 274/558] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Oct 20 14:29:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:07.242694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f01594-7337-4600-a479-72814177b589 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:07.324009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:07.397467 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2e20d-8ae3-4d4e-8f64-92cf7a4863de None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:07.430012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:07.438534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b820-b97d-47b2-8812-25a5054dc33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:07.440725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b820-b97d-47b2-8812-25a5054dc33b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:07.441209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b820-b97d-47b2-8812-25a5054dc33b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:07.441851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b820-b97d-47b2-8812-25a5054dc33b None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:07.443428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9b820-b97d-47b2-8812-25a5054dc33b None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:07.444129 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9b820-b97d-47b2-8812-25a5054dc33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:07.448488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9b820-b97d-47b2-8812-25a5054dc33b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:07.462776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 285/559] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:29:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:07.469241 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b820-b97d-47b2-8812-25a5054dc33b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:07.479416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f01594-7337-4600-a479-72814177b589 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:07.527021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a2701-96ab-4949-9bb3-38a5fd82abc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:07.527651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a2701-96ab-4949-9bb3-38a5fd82abc6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:07.527929 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a2701-96ab-4949-9bb3-38a5fd82abc6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:07.528206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a2701-96ab-4949-9bb3-38a5fd82abc6 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:07.528968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a2701-96ab-4949-9bb3-38a5fd82abc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:07.529276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a2701-96ab-4949-9bb3-38a5fd82abc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:07.533347 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a2701-96ab-4949-9bb3-38a5fd82abc6 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:07.570020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 275/560] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Oct 20 14:29:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:07.575911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a2701-96ab-4949-9bb3-38a5fd82abc6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:07.645359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b820-b97d-47b2-8812-25a5054dc33b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:07.705580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:07.712033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c767fb-5c3c-4ca7-8bc2-1eac39828018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:07.716415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c767fb-5c3c-4ca7-8bc2-1eac39828018 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:07.716415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c767fb-5c3c-4ca7-8bc2-1eac39828018 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:07.716415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c767fb-5c3c-4ca7-8bc2-1eac39828018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:07.723581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a2701-96ab-4949-9bb3-38a5fd82abc6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:07.727286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c767fb-5c3c-4ca7-8bc2-1eac39828018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:07.729420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c767fb-5c3c-4ca7-8bc2-1eac39828018 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-72c767fb-5c3c-4ca7-8bc2-1eac39828018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:07.745301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:07.746266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:07.747110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:07.747110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:07.747110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:07.747110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 286/561] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:07.751774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c767fb-5c3c-4ca7-8bc2-1eac39828018 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:07.805110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:07.817745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b23e3-ac11-41ed-893a-70ecc8be46e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:07.818388 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b23e3-ac11-41ed-893a-70ecc8be46e8 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:07.818666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b23e3-ac11-41ed-893a-70ecc8be46e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:07.818948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b23e3-ac11-41ed-893a-70ecc8be46e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:07.836219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b23e3-ac11-41ed-893a-70ecc8be46e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:07.837809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b23e3-ac11-41ed-893a-70ecc8be46e8 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-f44b23e3-ac11-41ed-893a-70ecc8be46e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:07.852731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:07.853689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:07.855253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:07.855253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:07.855253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:07.859437 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 276/562] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:07.861105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f1abd-99cc-4485-9f11-dc169d54fe82 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:07.861397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f1abd-99cc-4485-9f11-dc169d54fe82 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:07.861674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f1abd-99cc-4485-9f11-dc169d54fe82 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:07.863238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 277/563] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:07.868614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f1abd-99cc-4485-9f11-dc169d54fe82 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:07.870707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c767fb-5c3c-4ca7-8bc2-1eac39828018 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:07.909590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bec88-d8f1-4c7c-b0a1-80b0553a003f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:07.910202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bec88-d8f1-4c7c-b0a1-80b0553a003f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:07.910536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bec88-d8f1-4c7c-b0a1-80b0553a003f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:07.910819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bec88-d8f1-4c7c-b0a1-80b0553a003f None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:07.911469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bec88-d8f1-4c7c-b0a1-80b0553a003f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:07.911775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bec88-d8f1-4c7c-b0a1-80b0553a003f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:07.913183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bec88-d8f1-4c7c-b0a1-80b0553a003f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:07.926845 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 287/564] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:07.928095 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09305510-e051-43e0-9ee1-dab1bde498ff None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:07.928407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09305510-e051-43e0-9ee1-dab1bde498ff None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:07.928690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09305510-e051-43e0-9ee1-dab1bde498ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:07.949685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-09305510-e051-43e0-9ee1-dab1bde498ff None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:07.950969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-09305510-e051-43e0-9ee1-dab1bde498ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:08.002329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f1abd-99cc-4485-9f11-dc169d54fe82 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:08.033747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 288/565] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:08.034708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09305510-e051-43e0-9ee1-dab1bde498ff None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:08.061480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f27516-7a17-45c6-a918-751ba3640ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:08.062102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27516-7a17-45c6-a918-751ba3640ade None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.063228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27516-7a17-45c6-a918-751ba3640ade None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.063511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27516-7a17-45c6-a918-751ba3640ade None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.064951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f27516-7a17-45c6-a918-751ba3640ade None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:08.065263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f27516-7a17-45c6-a918-751ba3640ade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:08.066647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f27516-7a17-45c6-a918-751ba3640ade None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:08.089197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 278/566] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:08.095748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f27516-7a17-45c6-a918-751ba3640ade None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:08.097854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:08.101482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:08.214692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09305510-e051-43e0-9ee1-dab1bde498ff None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:08.279836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2695dcf-474f-4eb5-ac76-fab6b411cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:08.281893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f27516-7a17-45c6-a918-751ba3640ade None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:08.282687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2695dcf-474f-4eb5-ac76-fab6b411cf0a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.283114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2695dcf-474f-4eb5-ac76-fab6b411cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.283564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2695dcf-474f-4eb5-ac76-fab6b411cf0a None admin] PATH_INFO: `/v3/users/nova` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.298162 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2695dcf-474f-4eb5-ac76-fab6b411cf0a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:08.300052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2695dcf-474f-4eb5-ac76-fab6b411cf0a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-d2695dcf-474f-4eb5-ac76-fab6b411cf0a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:08.320218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:08.321485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:29:08.322781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:08.330261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 289/567] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Oct 20 14:29:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:08.336943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2695dcf-474f-4eb5-ac76-fab6b411cf0a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:08.357088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851ffd7-1073-49d0-a2ab-dda17107cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:08.357976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851ffd7-1073-49d0-a2ab-dda17107cf3b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.357976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851ffd7-1073-49d0-a2ab-dda17107cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.358270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851ffd7-1073-49d0-a2ab-dda17107cf3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.381100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851ffd7-1073-49d0-a2ab-dda17107cf3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:08.382580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851ffd7-1073-49d0-a2ab-dda17107cf3b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-8851ffd7-1073-49d0-a2ab-dda17107cf3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:08.402478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:08.403596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:08.404765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:08.404765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:08.404765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:08.404765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:08.404765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:08.404765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:08.404765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:08.404765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:08.412814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 279/568] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:08.419375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851ffd7-1073-49d0-a2ab-dda17107cf3b None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:08.453673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:08.460942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2695dcf-474f-4eb5-ac76-fab6b411cf0a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:08.486991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:08.501363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694791ba-2db7-4f0a-af5e-5b87a612dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:08.502420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694791ba-2db7-4f0a-af5e-5b87a612dc34 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.502855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694791ba-2db7-4f0a-af5e-5b87a612dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.503350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694791ba-2db7-4f0a-af5e-5b87a612dc34 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.504271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694791ba-2db7-4f0a-af5e-5b87a612dc34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:08.504837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694791ba-2db7-4f0a-af5e-5b87a612dc34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:08.506794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694791ba-2db7-4f0a-af5e-5b87a612dc34 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:08.528646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 290/569] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Oct 20 14:29:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:08.529903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694791ba-2db7-4f0a-af5e-5b87a612dc34 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:08.535765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851ffd7-1073-49d0-a2ab-dda17107cf3b None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:08.571576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b38825-524f-4324-b307-756f83414bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:08.572187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b38825-524f-4324-b307-756f83414bc7 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.573006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b38825-524f-4324-b307-756f83414bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.573797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b38825-524f-4324-b307-756f83414bc7 None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.574806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b38825-524f-4324-b307-756f83414bc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:08.574806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b38825-524f-4324-b307-756f83414bc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:08.576243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b38825-524f-4324-b307-756f83414bc7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:08.589362 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 280/570] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:08.590044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b38825-524f-4324-b307-756f83414bc7 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:08.638330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694791ba-2db7-4f0a-af5e-5b87a612dc34 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:08.673295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc16061-726d-425c-9b93-1caf2463b529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:08.673930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc16061-726d-425c-9b93-1caf2463b529 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.674213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc16061-726d-425c-9b93-1caf2463b529 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.674499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc16061-726d-425c-9b93-1caf2463b529 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.675079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc16061-726d-425c-9b93-1caf2463b529 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:08.676547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc16061-726d-425c-9b93-1caf2463b529 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-1dc16061-726d-425c-9b93-1caf2463b529 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:08.685542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:08.686527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:08.687408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 291/571] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Oct 20 14:29:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:08.688226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc16061-726d-425c-9b93-1caf2463b529 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:08.705446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b38825-524f-4324-b307-756f83414bc7 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:08.711269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:08.730443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:08.741528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1116dd-aa10-4100-bdec-8547766f4579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:08.742130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1116dd-aa10-4100-bdec-8547766f4579 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.742874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1116dd-aa10-4100-bdec-8547766f4579 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.742983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1116dd-aa10-4100-bdec-8547766f4579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.801329 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc16061-726d-425c-9b93-1caf2463b529 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:08.818656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1116dd-aa10-4100-bdec-8547766f4579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:08.820388 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1116dd-aa10-4100-bdec-8547766f4579 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:08.838683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9863c-7e08-4b63-8baa-3bf9ec01c28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:08.839337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9863c-7e08-4b63-8baa-3bf9ec01c28a None admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.839621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9863c-7e08-4b63-8baa-3bf9ec01c28a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.839887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9863c-7e08-4b63-8baa-3bf9ec01c28a None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/04f97b428a6347e39b9b1f97ace1e93b/roles/770887c5c808476d8505bf8b8fc5f27a` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.854886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9863c-7e08-4b63-8baa-3bf9ec01c28a None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=04f97b428a6347e39b9b1f97ace1e93b, role_id=770887c5c808476d8505bf8b8fc5f27a)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:08.856427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9863c-7e08-4b63-8baa-3bf9ec01c28a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:08.870231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 292/572] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:29:08 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/04f97b428a6347e39b9b1f97ace1e93b/roles/770887c5c808476d8505bf8b8fc5f27a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:29:08.871177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553181c1-c0cc-4fa2-b2a3-8a18278d78a3 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.871437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553181c1-c0cc-4fa2-b2a3-8a18278d78a3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.871717 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553181c1-c0cc-4fa2-b2a3-8a18278d78a3 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.876463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 293/573] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:08.876463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553181c1-c0cc-4fa2-b2a3-8a18278d78a3 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:08.973105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 281/574] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:29:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 14:29:08.975219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f4588-b76e-49f4-8d67-63fffc47e36b None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:08.975653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f4588-b76e-49f4-8d67-63fffc47e36b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:08.976069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f4588-b76e-49f4-8d67-63fffc47e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:08.988479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:09.006950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-3a9f4588-b76e-49f4-8d67-63fffc47e36b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:09.008844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3a9f4588-b76e-49f4-8d67-63fffc47e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:09.030390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553181c1-c0cc-4fa2-b2a3-8a18278d78a3 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.043238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:09.074168 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98904068-925d-495f-a95b-44b1a9cb6ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.075177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98904068-925d-495f-a95b-44b1a9cb6ac8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.075672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98904068-925d-495f-a95b-44b1a9cb6ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.076137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98904068-925d-495f-a95b-44b1a9cb6ac8 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.077015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98904068-925d-495f-a95b-44b1a9cb6ac8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:09.077634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98904068-925d-495f-a95b-44b1a9cb6ac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:09.079272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98904068-925d-495f-a95b-44b1a9cb6ac8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:09.085326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 294/575] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Oct 20 14:29:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:09.086626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98904068-925d-495f-a95b-44b1a9cb6ac8 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.143649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 282/576] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:09.148497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f4588-b76e-49f4-8d67-63fffc47e36b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.220411 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98904068-925d-495f-a95b-44b1a9cb6ac8 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.245741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:09.251209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dc45b-eee6-45b5-a971-33de0bb4ae5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.251993 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dc45b-eee6-45b5-a971-33de0bb4ae5f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.252282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dc45b-eee6-45b5-a971-33de0bb4ae5f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.252628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dc45b-eee6-45b5-a971-33de0bb4ae5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.263846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dc45b-eee6-45b5-a971-33de0bb4ae5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:09.265314 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dc45b-eee6-45b5-a971-33de0bb4ae5f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-1b7dc45b-eee6-45b5-a971-33de0bb4ae5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:09.280375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:09.281326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:09.282203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:09.282203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:09.282203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:09.282203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 295/577] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:09.283160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf779955-d28b-427d-acca-0bb2fbac15a3 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.283264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf779955-d28b-427d-acca-0bb2fbac15a3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.285512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf779955-d28b-427d-acca-0bb2fbac15a3 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.285512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 296/578] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:09.288184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f4588-b76e-49f4-8d67-63fffc47e36b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.289891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf779955-d28b-427d-acca-0bb2fbac15a3 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.347567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f94fa-e61a-4c1c-a42e-5531b7f3995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.348228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f94fa-e61a-4c1c-a42e-5531b7f3995b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.351639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f94fa-e61a-4c1c-a42e-5531b7f3995b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.351924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f94fa-e61a-4c1c-a42e-5531b7f3995b None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.352710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f94fa-e61a-4c1c-a42e-5531b7f3995b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:09.352918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f94fa-e61a-4c1c-a42e-5531b7f3995b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:09.355366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f94fa-e61a-4c1c-a42e-5531b7f3995b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:09.363075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 283/579] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:09.364486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6ad65-7349-4fee-b1e1-c41fda101710 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.364772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6ad65-7349-4fee-b1e1-c41fda101710 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.365044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6ad65-7349-4fee-b1e1-c41fda101710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.384250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-2df6ad65-7349-4fee-b1e1-c41fda101710 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:09.386135 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2df6ad65-7349-4fee-b1e1-c41fda101710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:09.400617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:09.415930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf779955-d28b-427d-acca-0bb2fbac15a3 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.451693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458b713-0466-4f50-ac53-1e26b09a97f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.452617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458b713-0466-4f50-ac53-1e26b09a97f7 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.453106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458b713-0466-4f50-ac53-1e26b09a97f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.453599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458b713-0466-4f50-ac53-1e26b09a97f7 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.454445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458b713-0466-4f50-ac53-1e26b09a97f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:09.454944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458b713-0466-4f50-ac53-1e26b09a97f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:09.456301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458b713-0466-4f50-ac53-1e26b09a97f7 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:09.464554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 284/580] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:09.464554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6ad65-7349-4fee-b1e1-c41fda101710 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.469446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 297/581] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:09.470371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458b713-0466-4f50-ac53-1e26b09a97f7 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.517196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:09.574749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458b713-0466-4f50-ac53-1e26b09a97f7 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.591991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6ad65-7349-4fee-b1e1-c41fda101710 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.610199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcac57-8d08-4b5a-946f-1dc8f0d127db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.610750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcac57-8d08-4b5a-946f-1dc8f0d127db None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.611028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcac57-8d08-4b5a-946f-1dc8f0d127db None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.611299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcac57-8d08-4b5a-946f-1dc8f0d127db None admin] PATH_INFO: `/v3/services/network` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.611926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcac57-8d08-4b5a-946f-1dc8f0d127db None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:09.613372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcac57-8d08-4b5a-946f-1dc8f0d127db None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-fcbcac57-8d08-4b5a-946f-1dc8f0d127db None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:09.623537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:09.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:09.625603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 298/582] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/services/network => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:09.626646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcac57-8d08-4b5a-946f-1dc8f0d127db None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.640811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a075f-f901-4b7f-a395-11f149ab283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.642154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a075f-f901-4b7f-a395-11f149ab283e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.642883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a075f-f901-4b7f-a395-11f149ab283e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.643346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a075f-f901-4b7f-a395-11f149ab283e None admin] PATH_INFO: `/v3/endpoints` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.644515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a075f-f901-4b7f-a395-11f149ab283e None admin] RBAC: Adding query filter params (interface=public, service_id=8699f617274145108f4b602cb5bb1ce6, region_id=RegionOne) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:09.645374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a075f-f901-4b7f-a395-11f149ab283e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:09.646937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a075f-f901-4b7f-a395-11f149ab283e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:09.659071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 285/583] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/endpoints?service_id=8699f617274145108f4b602cb5bb1ce6&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:09.660184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a075f-f901-4b7f-a395-11f149ab283e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.680365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:09.737233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcac57-8d08-4b5a-946f-1dc8f0d127db None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.749605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:09.781315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a892be6-428d-4ddd-8b7f-686cdc3bd693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.781952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a892be6-428d-4ddd-8b7f-686cdc3bd693 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.782234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a892be6-428d-4ddd-8b7f-686cdc3bd693 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.782510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a892be6-428d-4ddd-8b7f-686cdc3bd693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.786995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a075f-f901-4b7f-a395-11f149ab283e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.789920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a892be6-428d-4ddd-8b7f-686cdc3bd693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:09.791443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a892be6-428d-4ddd-8b7f-686cdc3bd693 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-4a892be6-428d-4ddd-8b7f-686cdc3bd693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:09.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:09.805420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 299/584] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:09.806302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a892be6-428d-4ddd-8b7f-686cdc3bd693 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.830134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c682c-70c7-4eb9-b03e-e116618f2ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.831276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c682c-70c7-4eb9-b03e-e116618f2ec3 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.832111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c682c-70c7-4eb9-b03e-e116618f2ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.832585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c682c-70c7-4eb9-b03e-e116618f2ec3 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.833101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c682c-70c7-4eb9-b03e-e116618f2ec3 None admin] RBAC: Adding query filter params (name=network) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:09.833405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c682c-70c7-4eb9-b03e-e116618f2ec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:09.834767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c682c-70c7-4eb9-b03e-e116618f2ec3 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:09.841207 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 286/585] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:09.842080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c682c-70c7-4eb9-b03e-e116618f2ec3 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.920319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a892be6-428d-4ddd-8b7f-686cdc3bd693 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.924637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:09.948732 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c682c-70c7-4eb9-b03e-e116618f2ec3 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:09.962911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9050e-7ec3-4cbf-9610-3147d32b087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.964366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9050e-7ec3-4cbf-9610-3147d32b087f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.964935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9050e-7ec3-4cbf-9610-3147d32b087f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.965388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9050e-7ec3-4cbf-9610-3147d32b087f None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:09.966217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9050e-7ec3-4cbf-9610-3147d32b087f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:09.967775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9050e-7ec3-4cbf-9610-3147d32b087f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:09.973506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 300/586] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:09.974729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9050e-7ec3-4cbf-9610-3147d32b087f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:09.993772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece9ee9-3522-4136-846b-a2c5af2db065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:09.995452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece9ee9-3522-4136-846b-a2c5af2db065 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:09.995452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece9ee9-3522-4136-846b-a2c5af2db065 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:09.995652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece9ee9-3522-4136-846b-a2c5af2db065 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:10.007248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece9ee9-3522-4136-846b-a2c5af2db065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-1ece9ee9-3522-4136-846b-a2c5af2db065 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:10.013269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:10.014255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:10.015058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:10.019560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 287/587] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:10.020615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece9ee9-3522-4136-846b-a2c5af2db065 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:10.023080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:10.108643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9050e-7ec3-4cbf-9610-3147d32b087f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:10.147964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83ad93-ef16-4c36-bc20-57ae9a82bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:10.148936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83ad93-ef16-4c36-bc20-57ae9a82bc61 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:10.149126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece9ee9-3522-4136-846b-a2c5af2db065 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:10.149591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83ad93-ef16-4c36-bc20-57ae9a82bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:10.150037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83ad93-ef16-4c36-bc20-57ae9a82bc61 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:10.151762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83ad93-ef16-4c36-bc20-57ae9a82bc61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:10.152283 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83ad93-ef16-4c36-bc20-57ae9a82bc61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:10.153962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83ad93-ef16-4c36-bc20-57ae9a82bc61 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:10.170894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 301/588] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:10.172082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83ad93-ef16-4c36-bc20-57ae9a82bc61 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:10.197098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001507a-3cf5-4cd9-90c8-d806649d3a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:10.198313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001507a-3cf5-4cd9-90c8-d806649d3a71 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:10.199151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001507a-3cf5-4cd9-90c8-d806649d3a71 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:10.199151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001507a-3cf5-4cd9-90c8-d806649d3a71 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:10.199484 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001507a-3cf5-4cd9-90c8-d806649d3a71 None admin] RBAC: Adding query filter params (type=network) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:10.199790 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001507a-3cf5-4cd9-90c8-d806649d3a71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:10.201339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001507a-3cf5-4cd9-90c8-d806649d3a71 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:10.203696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:10.212262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 288/589] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:10.213651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001507a-3cf5-4cd9-90c8-d806649d3a71 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:10.280753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:10.295753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83ad93-ef16-4c36-bc20-57ae9a82bc61 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:10.344004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf80a6-621f-44d9-bba2-6533471bd71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:10.345107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf80a6-621f-44d9-bba2-6533471bd71b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:10.345555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf80a6-621f-44d9-bba2-6533471bd71b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:10.345965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf80a6-621f-44d9-bba2-6533471bd71b None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:10.346667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001507a-3cf5-4cd9-90c8-d806649d3a71 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:10.346931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf80a6-621f-44d9-bba2-6533471bd71b None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:10.347385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf80a6-621f-44d9-bba2-6533471bd71b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:10.349213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf80a6-621f-44d9-bba2-6533471bd71b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:10.361693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 302/590] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:10.363126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf80a6-621f-44d9-bba2-6533471bd71b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:10.389604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9342e0-d899-4e18-8909-2cdcee6a3145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:10.390352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9342e0-d899-4e18-8909-2cdcee6a3145 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:10.390727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9342e0-d899-4e18-8909-2cdcee6a3145 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:10.391114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9342e0-d899-4e18-8909-2cdcee6a3145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:10.391932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9342e0-d899-4e18-8909-2cdcee6a3145 None admin] RBAC: Adding query filter params (role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=9647f8caa6cd4208800e2371fdf28301) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:10.392338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9342e0-d899-4e18-8909-2cdcee6a3145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:10.393990 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9342e0-d899-4e18-8909-2cdcee6a3145 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:10.404719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 289/591] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:29:10 2022] GET /identity/v3/role_assignments?user.id=9647f8caa6cd4208800e2371fdf28301&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:10.404719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9342e0-d899-4e18-8909-2cdcee6a3145 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:10.525311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:10.543165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf80a6-621f-44d9-bba2-6533471bd71b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:10.601486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9342e0-d899-4e18-8909-2cdcee6a3145 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:10.614348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf775af-18c8-45a2-a724-b242ee6c346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:10.615011 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf775af-18c8-45a2-a724-b242ee6c346d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:10.615290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf775af-18c8-45a2-a724-b242ee6c346d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:10.615558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf775af-18c8-45a2-a724-b242ee6c346d None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:10.617730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf775af-18c8-45a2-a724-b242ee6c346d None admin] RBAC: Adding query filter params (interface=public, service_id=53ebbc820ddc4538945f11775c8d516d, region_id=RegionOne) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:10.618044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf775af-18c8-45a2-a724-b242ee6c346d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:10.619471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf775af-18c8-45a2-a724-b242ee6c346d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:10.631335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 303/592] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Oct 20 14:29:10 2022] GET /identity/v3/endpoints?service_id=53ebbc820ddc4538945f11775c8d516d&interface=public®ion_id=RegionOne => generated 194 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:10.637579 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf775af-18c8-45a2-a724-b242ee6c346d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:10.646622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af8809-b641-4275-895d-369238886c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:10.647562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af8809-b641-4275-895d-369238886c50 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:10.648036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af8809-b641-4275-895d-369238886c50 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:10.648534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af8809-b641-4275-895d-369238886c50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:10.659609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af8809-b641-4275-895d-369238886c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:10.661374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af8809-b641-4275-895d-369238886c50 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:10.667823 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-b7af8809-b641-4275-895d-369238886c50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:10.682019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:10.682980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:10.684291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:10.684291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:10.684291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:10.684291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 290/593] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:10.689540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af8809-b641-4275-895d-369238886c50 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:10.770911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf775af-18c8-45a2-a724-b242ee6c346d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:10.805897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af8809-b641-4275-895d-369238886c50 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:10.809048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195924e-ce4f-450a-b435-f36b36872f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:10.809661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195924e-ce4f-450a-b435-f36b36872f41 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:10.809939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195924e-ce4f-450a-b435-f36b36872f41 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:10.810207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195924e-ce4f-450a-b435-f36b36872f41 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:10.810807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195924e-ce4f-450a-b435-f36b36872f41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:10.812189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195924e-ce4f-450a-b435-f36b36872f41 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:10.820423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 304/594] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Oct 20 14:29:10 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:10.828907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:10.854573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c303a2-e664-43df-b3be-f5a2fe754386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:10.855777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c303a2-e664-43df-b3be-f5a2fe754386 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:10.856254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c303a2-e664-43df-b3be-f5a2fe754386 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:10.856787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c303a2-e664-43df-b3be-f5a2fe754386 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:10.857677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c303a2-e664-43df-b3be-f5a2fe754386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:10.858247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c303a2-e664-43df-b3be-f5a2fe754386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:10.859861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c303a2-e664-43df-b3be-f5a2fe754386 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:10.875007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 291/595] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:10.880287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195924e-ce4f-450a-b435-f36b36872f41 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:10.972835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:10.993198 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195924e-ce4f-450a-b435-f36b36872f41 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:11.037916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f70d50-e198-49de-89b7-3bd78959630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:11.038557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f70d50-e198-49de-89b7-3bd78959630f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:11.038834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f70d50-e198-49de-89b7-3bd78959630f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:11.039105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f70d50-e198-49de-89b7-3bd78959630f None admin] PATH_INFO: `/v3/users/swift` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:11.047909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f70d50-e198-49de-89b7-3bd78959630f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:11.049405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f70d50-e198-49de-89b7-3bd78959630f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-54f70d50-e198-49de-89b7-3bd78959630f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:11.062228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:11.063161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:11.064527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 305/596] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:29:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:11.069091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c303a2-e664-43df-b3be-f5a2fe754386 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:11.197970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c303a2-e664-43df-b3be-f5a2fe754386 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:11.249664 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75591b3c-0f72-4884-b23e-600252ccce59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:11.250292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75591b3c-0f72-4884-b23e-600252ccce59 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:11.250573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75591b3c-0f72-4884-b23e-600252ccce59 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:11.250860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75591b3c-0f72-4884-b23e-600252ccce59 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:11.251593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75591b3c-0f72-4884-b23e-600252ccce59 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:11.251902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75591b3c-0f72-4884-b23e-600252ccce59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:11.253381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75591b3c-0f72-4884-b23e-600252ccce59 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:11.272508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 292/597] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Oct 20 14:29:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:11.278310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f70d50-e198-49de-89b7-3bd78959630f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:11.323241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81df8fe-1baf-4357-a78f-8ebe29bc8702 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:11.323841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81df8fe-1baf-4357-a78f-8ebe29bc8702 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:11.323956 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81df8fe-1baf-4357-a78f-8ebe29bc8702 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:11.325674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 293/598] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:11.331976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fe59a-7c1b-4b3f-b7c9-ce662e7336ad None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:11.332267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fe59a-7c1b-4b3f-b7c9-ce662e7336ad None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:11.332578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fe59a-7c1b-4b3f-b7c9-ce662e7336ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:11.365469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-550fe59a-7c1b-4b3f-b7c9-ce662e7336ad None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:11.366673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-550fe59a-7c1b-4b3f-b7c9-ce662e7336ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:11.425878 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:11.454846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f70d50-e198-49de-89b7-3bd78959630f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:11.495810 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 294/599] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:11.533229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0db683-0cdc-4c9a-a2e5-9905d8b1c4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:11.533844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0db683-0cdc-4c9a-a2e5-9905d8b1c4ed None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:11.534122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0db683-0cdc-4c9a-a2e5-9905d8b1c4ed None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:11.534395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0db683-0cdc-4c9a-a2e5-9905d8b1c4ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:11.557953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:11.630925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0db683-0cdc-4c9a-a2e5-9905d8b1c4ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:11.632728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0db683-0cdc-4c9a-a2e5-9905d8b1c4ed None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:11.750100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 306/600] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:29:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 14:29:11.755952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fe59a-7c1b-4b3f-b7c9-ce662e7336ad None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:11.866634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0db683-0cdc-4c9a-a2e5-9905d8b1c4ed None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:11.915375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fe59a-7c1b-4b3f-b7c9-ce662e7336ad None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:11.940987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:11.988412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18d5e0-57d2-4c29-b5d6-fe4e3aae23cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:11.989416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18d5e0-57d2-4c29-b5d6-fe4e3aae23cc None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:11.989888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18d5e0-57d2-4c29-b5d6-fe4e3aae23cc None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:11.990368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18d5e0-57d2-4c29-b5d6-fe4e3aae23cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.011446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18d5e0-57d2-4c29-b5d6-fe4e3aae23cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:12.014973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18d5e0-57d2-4c29-b5d6-fe4e3aae23cc None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-ae18d5e0-57d2-4c29-b5d6-fe4e3aae23cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:12.039820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:12.040789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:12.041805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:12.041805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:12.041805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:12.047506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 295/601] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:12.053152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18d5e0-57d2-4c29-b5d6-fe4e3aae23cc None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:12.054013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0db683-0cdc-4c9a-a2e5-9905d8b1c4ed None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:12.089615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:12.129038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194281c-bb7c-44a5-8120-69a3a2411c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:12.130981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194281c-bb7c-44a5-8120-69a3a2411c5e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:12.131489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194281c-bb7c-44a5-8120-69a3a2411c5e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:12.133304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194281c-bb7c-44a5-8120-69a3a2411c5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.152923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194281c-bb7c-44a5-8120-69a3a2411c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-5194281c-bb7c-44a5-8120-69a3a2411c5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:12.160431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:12.161328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:29:12.162187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:12.166491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 307/602] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:29:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:12.172731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194281c-bb7c-44a5-8120-69a3a2411c5e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:12.239170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18d5e0-57d2-4c29-b5d6-fe4e3aae23cc None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:12.297891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194281c-bb7c-44a5-8120-69a3a2411c5e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:12.313711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:12.317615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3582e-6ae7-40a5-8756-852863dbdf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:12.318245 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3582e-6ae7-40a5-8756-852863dbdf67 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:12.318530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3582e-6ae7-40a5-8756-852863dbdf67 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:12.318811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3582e-6ae7-40a5-8756-852863dbdf67 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.319474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3582e-6ae7-40a5-8756-852863dbdf67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:12.319765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3582e-6ae7-40a5-8756-852863dbdf67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:12.321294 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3582e-6ae7-40a5-8756-852863dbdf67 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:12.336102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c864bef-bd96-47ca-980c-c032022371ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:12.343429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c864bef-bd96-47ca-980c-c032022371ce None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:12.343983 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c864bef-bd96-47ca-980c-c032022371ce None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:12.344503 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c864bef-bd96-47ca-980c-c032022371ce None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.345402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c864bef-bd96-47ca-980c-c032022371ce None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:12.345939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c864bef-bd96-47ca-980c-c032022371ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:12.348037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c864bef-bd96-47ca-980c-c032022371ce None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:12.358560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 296/603] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:12.362060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 308/604] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:29:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:12.369093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3582e-6ae7-40a5-8756-852863dbdf67 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:12.373700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c864bef-bd96-47ca-980c-c032022371ce None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:12.484500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c864bef-bd96-47ca-980c-c032022371ce None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:12.519962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd420a24-9001-4680-9719-8978fe4e6390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:12.520995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd420a24-9001-4680-9719-8978fe4e6390 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:12.521515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd420a24-9001-4680-9719-8978fe4e6390 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:12.521999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd420a24-9001-4680-9719-8978fe4e6390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.529541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd420a24-9001-4680-9719-8978fe4e6390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:12.531244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd420a24-9001-4680-9719-8978fe4e6390 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-bd420a24-9001-4680-9719-8978fe4e6390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:12.543066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:12.544093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:12.545381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:12.545381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:12.545381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:12.545381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:12.545381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:12.545381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:12.545381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:12.545381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:12.551941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 309/605] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:12.553384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd420a24-9001-4680-9719-8978fe4e6390 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:12.564729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:12.580734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:12.600057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3582e-6ae7-40a5-8756-852863dbdf67 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:12.643624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d14370-9c0f-45c0-9791-143e4ab04c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:12.644257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d14370-9c0f-45c0-9791-143e4ab04c83 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:12.644583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d14370-9c0f-45c0-9791-143e4ab04c83 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:12.644856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d14370-9c0f-45c0-9791-143e4ab04c83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.655513 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d14370-9c0f-45c0-9791-143e4ab04c83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:12.657011 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d14370-9c0f-45c0-9791-143e4ab04c83 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:12.660133 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd420a24-9001-4680-9719-8978fe4e6390 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-02d14370-9c0f-45c0-9791-143e4ab04c83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:12.671797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:12.672808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:12.673752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:12.673752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:12.673752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:12.676678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 297/606] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:12.677937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9b3ff-742f-4c60-afd7-66db2f828a53 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:12.678226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9b3ff-742f-4c60-afd7-66db2f828a53 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:12.678507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9b3ff-742f-4c60-afd7-66db2f828a53 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.681209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 298/607] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:12.682461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9b3ff-742f-4c60-afd7-66db2f828a53 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:12.700564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84be06-1f49-4a61-9c88-136d61adaaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:12.701539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84be06-1f49-4a61-9c88-136d61adaaf9 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:12.702022 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84be06-1f49-4a61-9c88-136d61adaaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:12.702511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84be06-1f49-4a61-9c88-136d61adaaf9 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.703396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84be06-1f49-4a61-9c88-136d61adaaf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:12.703921 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84be06-1f49-4a61-9c88-136d61adaaf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:12.705602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84be06-1f49-4a61-9c88-136d61adaaf9 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:12.723141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 310/608] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:12.723141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb599b-cbd7-401f-88ad-1ae3f0ef4ac4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:12.723141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb599b-cbd7-401f-88ad-1ae3f0ef4ac4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:12.723141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb599b-cbd7-401f-88ad-1ae3f0ef4ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.743272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-08cb599b-cbd7-401f-88ad-1ae3f0ef4ac4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:12.745271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-08cb599b-cbd7-401f-88ad-1ae3f0ef4ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:12.808696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9b3ff-742f-4c60-afd7-66db2f828a53 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:12.832481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 311/609] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:12.834482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb599b-cbd7-401f-88ad-1ae3f0ef4ac4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:12.863540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:12.867908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:12.891362 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca2d61-8a9e-4bb2-ac60-0c19f2603e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:12.891978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca2d61-8a9e-4bb2-ac60-0c19f2603e86 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:12.892259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca2d61-8a9e-4bb2-ac60-0c19f2603e86 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:12.893365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca2d61-8a9e-4bb2-ac60-0c19f2603e86 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:12.893972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca2d61-8a9e-4bb2-ac60-0c19f2603e86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:12.894281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca2d61-8a9e-4bb2-ac60-0c19f2603e86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:12.895673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca2d61-8a9e-4bb2-ac60-0c19f2603e86 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:12.921592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 299/610] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:12.926778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca2d61-8a9e-4bb2-ac60-0c19f2603e86 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:13.026712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb599b-cbd7-401f-88ad-1ae3f0ef4ac4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:13.095066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190555cf-5e03-42fe-a68d-4c8b4e57dca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:13.095677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190555cf-5e03-42fe-a68d-4c8b4e57dca9 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.095955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190555cf-5e03-42fe-a68d-4c8b4e57dca9 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.096230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190555cf-5e03-42fe-a68d-4c8b4e57dca9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.096984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190555cf-5e03-42fe-a68d-4c8b4e57dca9 None admin] RBAC: Adding query filter params (role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=61d6c1cd010d4fa387fa64864419bc6a) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:13.097292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190555cf-5e03-42fe-a68d-4c8b4e57dca9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:13.098732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190555cf-5e03-42fe-a68d-4c8b4e57dca9 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:13.111420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 312/611] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:29:12 2022] GET /identity/v3/role_assignments?user.id=61d6c1cd010d4fa387fa64864419bc6a&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:13.117000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca2d61-8a9e-4bb2-ac60-0c19f2603e86 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:13.161414 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190555cf-5e03-42fe-a68d-4c8b4e57dca9 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:13.163458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaeb37-9df3-4f2f-8663-471a03bb3615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:13.164239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaeb37-9df3-4f2f-8663-471a03bb3615 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.164557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaeb37-9df3-4f2f-8663-471a03bb3615 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.164837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaeb37-9df3-4f2f-8663-471a03bb3615 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.177166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaeb37-9df3-4f2f-8663-471a03bb3615 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:13.178651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaeb37-9df3-4f2f-8663-471a03bb3615 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-49eaeb37-9df3-4f2f-8663-471a03bb3615 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:13.202763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:13.203772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:13.205968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 300/612] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Oct 20 14:29:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:13.210081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaeb37-9df3-4f2f-8663-471a03bb3615 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:13.273546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:13.292882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190555cf-5e03-42fe-a68d-4c8b4e57dca9 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:13.297478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:13.344444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79eec9a-77c0-44ba-a8e1-b1214934e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:13.345505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79eec9a-77c0-44ba-a8e1-b1214934e543 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.345789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79eec9a-77c0-44ba-a8e1-b1214934e543 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.346351 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79eec9a-77c0-44ba-a8e1-b1214934e543 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.346943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79eec9a-77c0-44ba-a8e1-b1214934e543 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:13.353603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79eec9a-77c0-44ba-a8e1-b1214934e543 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-f79eec9a-77c0-44ba-a8e1-b1214934e543 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:13.366358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:13.367693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:13.368605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 313/613] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Oct 20 14:29:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:13.374421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20668c16-95a1-4e47-aee3-91fbe8c2a413 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.374730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20668c16-95a1-4e47-aee3-91fbe8c2a413 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.375267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20668c16-95a1-4e47-aee3-91fbe8c2a413 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.376516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 314/614] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:13.377107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20668c16-95a1-4e47-aee3-91fbe8c2a413 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:13.381269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaeb37-9df3-4f2f-8663-471a03bb3615 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:13.425953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72224e9-7ae5-4acf-9c8d-afb3fd3fdfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:13.426696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72224e9-7ae5-4acf-9c8d-afb3fd3fdfa1 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.426980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72224e9-7ae5-4acf-9c8d-afb3fd3fdfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.427255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72224e9-7ae5-4acf-9c8d-afb3fd3fdfa1 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.428985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72224e9-7ae5-4acf-9c8d-afb3fd3fdfa1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:13.428985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72224e9-7ae5-4acf-9c8d-afb3fd3fdfa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:13.429952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72224e9-7ae5-4acf-9c8d-afb3fd3fdfa1 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:13.451648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 301/615] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Oct 20 14:29:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:13.456782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd0466-67c6-43ce-ae5b-5f5694910936 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.456782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd0466-67c6-43ce-ae5b-5f5694910936 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.456782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd0466-67c6-43ce-ae5b-5f5694910936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.473094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-56dd0466-67c6-43ce-ae5b-5f5694910936 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:13.474364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-56dd0466-67c6-43ce-ae5b-5f5694910936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:13.522132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20668c16-95a1-4e47-aee3-91fbe8c2a413 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:13.547288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 302/616] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:13.548057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd0466-67c6-43ce-ae5b-5f5694910936 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:13.570025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41885d5-8e29-4ec0-a44e-2bc38bc7b18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:13.570666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41885d5-8e29-4ec0-a44e-2bc38bc7b18c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.570945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41885d5-8e29-4ec0-a44e-2bc38bc7b18c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.571215 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41885d5-8e29-4ec0-a44e-2bc38bc7b18c None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.571848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41885d5-8e29-4ec0-a44e-2bc38bc7b18c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:13.572151 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41885d5-8e29-4ec0-a44e-2bc38bc7b18c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:13.573625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41885d5-8e29-4ec0-a44e-2bc38bc7b18c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:13.579866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 315/617] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Oct 20 14:29:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:13.581315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75f08-4c69-4ca0-a353-e57aadfb912c None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.581595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75f08-4c69-4ca0-a353-e57aadfb912c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.581870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75f08-4c69-4ca0-a353-e57aadfb912c None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.583183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 316/618] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:13.588903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c75f08-4c69-4ca0-a353-e57aadfb912c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:13.595914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:13.622657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:13.676027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd0466-67c6-43ce-ae5b-5f5694910936 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:13.709746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c75f08-4c69-4ca0-a353-e57aadfb912c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:13.718773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bd621-f097-418b-92f3-487fe56fc567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:13.719393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bd621-f097-418b-92f3-487fe56fc567 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.719686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bd621-f097-418b-92f3-487fe56fc567 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.719950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bd621-f097-418b-92f3-487fe56fc567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.748868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53837fdf-0ea9-455b-8892-01d0e618624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:13.750016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53837fdf-0ea9-455b-8892-01d0e618624b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.750299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53837fdf-0ea9-455b-8892-01d0e618624b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.750675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53837fdf-0ea9-455b-8892-01d0e618624b None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.751366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53837fdf-0ea9-455b-8892-01d0e618624b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:13.751733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53837fdf-0ea9-455b-8892-01d0e618624b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:13.753437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53837fdf-0ea9-455b-8892-01d0e618624b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:13.760399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 317/619] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Oct 20 14:29:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:13.761687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b263e0-d2d3-469a-9f57-5494496d6c8f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:13.762035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b263e0-d2d3-469a-9f57-5494496d6c8f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:13.762380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b263e0-d2d3-469a-9f57-5494496d6c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:13.782967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-a7b263e0-d2d3-469a-9f57-5494496d6c8f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:13.784982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a7b263e0-d2d3-469a-9f57-5494496d6c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:13.803894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bd621-f097-418b-92f3-487fe56fc567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:13.806228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bd621-f097-418b-92f3-487fe56fc567 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:13.851468 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:13.860997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:13.872681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 318/620] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:13.873563 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b263e0-d2d3-469a-9f57-5494496d6c8f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:13.922929 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 303/621] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:29:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 14:29:13.924216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bd621-f097-418b-92f3-487fe56fc567 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.002844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b263e0-d2d3-469a-9f57-5494496d6c8f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.039561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728a57a-a8b8-4bb7-ab0c-c03d0425b744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.040576 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a57a-a8b8-4bb7-ab0c-c03d0425b744 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.041144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a57a-a8b8-4bb7-ab0c-c03d0425b744 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.042180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a57a-a8b8-4bb7-ab0c-c03d0425b744 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.043052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728a57a-a8b8-4bb7-ab0c-c03d0425b744 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.044642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728a57a-a8b8-4bb7-ab0c-c03d0425b744 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.048419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bd621-f097-418b-92f3-487fe56fc567 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.071905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 319/622] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:29:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:14.073903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728a57a-a8b8-4bb7-ab0c-c03d0425b744 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.107016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:14.115026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97abf64-cbda-4625-ba35-2532e116fd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.117683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97abf64-cbda-4625-ba35-2532e116fd4c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.119050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97abf64-cbda-4625-ba35-2532e116fd4c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.121901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97abf64-cbda-4625-ba35-2532e116fd4c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.122823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97abf64-cbda-4625-ba35-2532e116fd4c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.124617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97abf64-cbda-4625-ba35-2532e116fd4c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-f97abf64-cbda-4625-ba35-2532e116fd4c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:14.145247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:14.146241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 14:29:14.147107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:14.155927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 304/623] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Oct 20 14:29:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:14.157343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97abf64-cbda-4625-ba35-2532e116fd4c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.184740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:14.236049 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728a57a-a8b8-4bb7-ab0c-c03d0425b744 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.286735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97abf64-cbda-4625-ba35-2532e116fd4c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.288305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e3cab-35fb-473a-9f73-de294ff12ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.289248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e3cab-35fb-473a-9f73-de294ff12ea1 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.289530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e3cab-35fb-473a-9f73-de294ff12ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.290586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e3cab-35fb-473a-9f73-de294ff12ea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.304982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e3cab-35fb-473a-9f73-de294ff12ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.307232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e3cab-35fb-473a-9f73-de294ff12ea1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-b54e3cab-35fb-473a-9f73-de294ff12ea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:14.326643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:14.327597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:14.328489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:14.328489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:14.328489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:14.328489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 320/624] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:14.329378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e3cab-35fb-473a-9f73-de294ff12ea1 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.335631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192c19-7bfa-4b32-affb-4642902071e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.336280 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192c19-7bfa-4b32-affb-4642902071e4 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.340169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192c19-7bfa-4b32-affb-4642902071e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.340169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192c19-7bfa-4b32-affb-4642902071e4 None admin] PATH_INFO: `/v3/services/network` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.340169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8192c19-7bfa-4b32-affb-4642902071e4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.340169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8192c19-7bfa-4b32-affb-4642902071e4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-c8192c19-7bfa-4b32-affb-4642902071e4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:14.352603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:14.353713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:14.354522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 305/625] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:14.355321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192c19-7bfa-4b32-affb-4642902071e4 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.425623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:14.438839 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e3cab-35fb-473a-9f73-de294ff12ea1 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.449108 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:14.474548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192c19-7bfa-4b32-affb-4642902071e4 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.474789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbede2-c949-4449-9fa4-baa8a0de50ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.475271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbede2-c949-4449-9fa4-baa8a0de50ad None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.475551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbede2-c949-4449-9fa4-baa8a0de50ad None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.475819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbede2-c949-4449-9fa4-baa8a0de50ad None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.476475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbede2-c949-4449-9fa4-baa8a0de50ad None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:14.476776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbede2-c949-4449-9fa4-baa8a0de50ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.479311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbede2-c949-4449-9fa4-baa8a0de50ad None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.483612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 321/626] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:14.484625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbede2-c949-4449-9fa4-baa8a0de50ad None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.515085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e5ef0-f651-4705-b714-47330b0a4e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.516142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e5ef0-f651-4705-b714-47330b0a4e16 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.516704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e5ef0-f651-4705-b714-47330b0a4e16 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.517203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e5ef0-f651-4705-b714-47330b0a4e16 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.518058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e5ef0-f651-4705-b714-47330b0a4e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:14.518587 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e5ef0-f651-4705-b714-47330b0a4e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.520391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e5ef0-f651-4705-b714-47330b0a4e16 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.533814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 306/627] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:14.535000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e5ef0-f651-4705-b714-47330b0a4e16 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.591238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbede2-c949-4449-9fa4-baa8a0de50ad None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.625581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4e72e-8923-432f-8739-4e7ec3e830b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.626186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4e72e-8923-432f-8739-4e7ec3e830b0 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.626463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4e72e-8923-432f-8739-4e7ec3e830b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.626739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4e72e-8923-432f-8739-4e7ec3e830b0 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.627420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4e72e-8923-432f-8739-4e7ec3e830b0 None admin] RBAC: Adding query filter params (name=network) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:14.627649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4e72e-8923-432f-8739-4e7ec3e830b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.629038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4e72e-8923-432f-8739-4e7ec3e830b0 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.638925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 322/628] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:14.638925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4e72e-8923-432f-8739-4e7ec3e830b0 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.653460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e5ef0-f651-4705-b714-47330b0a4e16 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.670816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:14.678253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:14.695200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432883a4-de65-4765-ae3d-b1d2073bb0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.696200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432883a4-de65-4765-ae3d-b1d2073bb0cb None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.696715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432883a4-de65-4765-ae3d-b1d2073bb0cb None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.697768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432883a4-de65-4765-ae3d-b1d2073bb0cb None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.698610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432883a4-de65-4765-ae3d-b1d2073bb0cb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:14.699139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432883a4-de65-4765-ae3d-b1d2073bb0cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.700943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432883a4-de65-4765-ae3d-b1d2073bb0cb None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.706266 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 307/629] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:14.707435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432883a4-de65-4765-ae3d-b1d2073bb0cb None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.740989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4e72e-8923-432f-8739-4e7ec3e830b0 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.781005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce11db-2f0d-4530-9baa-762c70e856a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.781688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce11db-2f0d-4530-9baa-762c70e856a1 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.781969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce11db-2f0d-4530-9baa-762c70e856a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.782238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce11db-2f0d-4530-9baa-762c70e856a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.790176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce11db-2f0d-4530-9baa-762c70e856a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.791466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce11db-2f0d-4530-9baa-762c70e856a1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-d5ce11db-2f0d-4530-9baa-762c70e856a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:14.803716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:14.804701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:14.805567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:14.805567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:14.805567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:14.805567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:14.805567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:14.805567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:14.805567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:14.805567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:14.805567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 323/630] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:14.811971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce11db-2f0d-4530-9baa-762c70e856a1 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.827634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432883a4-de65-4765-ae3d-b1d2073bb0cb None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.866815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163522ea-acd3-459a-8534-290f04a55dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.868117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163522ea-acd3-459a-8534-290f04a55dd9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.868644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163522ea-acd3-459a-8534-290f04a55dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.869662 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163522ea-acd3-459a-8534-290f04a55dd9 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.870740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163522ea-acd3-459a-8534-290f04a55dd9 None admin] RBAC: Adding query filter params (type=network) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:14.871365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163522ea-acd3-459a-8534-290f04a55dd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.873123 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163522ea-acd3-459a-8534-290f04a55dd9 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.879446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 308/631] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:14.885482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163522ea-acd3-459a-8534-290f04a55dd9 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:14.917357 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:14.923703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce11db-2f0d-4530-9baa-762c70e856a1 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:14.943197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:14.965473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba46cec-37db-489c-a141-ccc28ad5578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:14.966109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba46cec-37db-489c-a141-ccc28ad5578c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:14.966392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba46cec-37db-489c-a141-ccc28ad5578c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:14.966665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba46cec-37db-489c-a141-ccc28ad5578c None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:14.967349 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba46cec-37db-489c-a141-ccc28ad5578c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:14.967659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba46cec-37db-489c-a141-ccc28ad5578c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:14.969243 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba46cec-37db-489c-a141-ccc28ad5578c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:14.983084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 324/632] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:14.989058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba46cec-37db-489c-a141-ccc28ad5578c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:15.007323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163522ea-acd3-459a-8534-290f04a55dd9 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:15.046104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03503e3-db61-4239-907b-c9dfe4bb18fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:15.047083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03503e3-db61-4239-907b-c9dfe4bb18fb None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:15.047670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03503e3-db61-4239-907b-c9dfe4bb18fb None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:15.048144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03503e3-db61-4239-907b-c9dfe4bb18fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:15.049192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03503e3-db61-4239-907b-c9dfe4bb18fb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:15.050999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03503e3-db61-4239-907b-c9dfe4bb18fb None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:15.082929 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 309/633] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:29:14 2022] POST /identity/v3/endpoints => generated 337 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:29:15.099568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba46cec-37db-489c-a141-ccc28ad5578c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:15.146093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40e256-5cab-402d-856d-bbf36f9c27ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:15.146609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40e256-5cab-402d-856d-bbf36f9c27ff None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:15.146884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40e256-5cab-402d-856d-bbf36f9c27ff None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:15.147152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40e256-5cab-402d-856d-bbf36f9c27ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:15.147877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40e256-5cab-402d-856d-bbf36f9c27ff None admin] RBAC: Adding query filter params (role.id=770887c5c808476d8505bf8b8fc5f27a, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=04f97b428a6347e39b9b1f97ace1e93b) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:15.148181 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40e256-5cab-402d-856d-bbf36f9c27ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:15.149663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40e256-5cab-402d-856d-bbf36f9c27ff None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:15.157670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 325/634] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:29:14 2022] GET /identity/v3/role_assignments?user.id=04f97b428a6347e39b9b1f97ace1e93b&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=770887c5c808476d8505bf8b8fc5f27a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:15.460499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb010b97-0627-4ff0-9727-086a5923523c None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:15.461511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb010b97-0627-4ff0-9727-086a5923523c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:15.462017 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb010b97-0627-4ff0-9727-086a5923523c None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:15.463914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 310/635] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:15.469473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0317f9b-940d-4b80-82cf-76a826a5dfa9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:15.469758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0317f9b-940d-4b80-82cf-76a826a5dfa9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:15.470030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0317f9b-940d-4b80-82cf-76a826a5dfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:15.490715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-b0317f9b-940d-4b80-82cf-76a826a5dfa9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:15.491901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b0317f9b-940d-4b80-82cf-76a826a5dfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:15.504500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:15.578512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 326/636] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:15.857403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb010b97-0627-4ff0-9727-086a5923523c None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:15.964816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb010b97-0627-4ff0-9727-086a5923523c None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:15.967207 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:16.002469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4896a7-6332-4d4a-9a36-26ce1acb5b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:16.003104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4896a7-6332-4d4a-9a36-26ce1acb5b48 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:16.003383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4896a7-6332-4d4a-9a36-26ce1acb5b48 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:16.003657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4896a7-6332-4d4a-9a36-26ce1acb5b48 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:16.004243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4896a7-6332-4d4a-9a36-26ce1acb5b48 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:16.005752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4896a7-6332-4d4a-9a36-26ce1acb5b48 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-7d4896a7-6332-4d4a-9a36-26ce1acb5b48 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:16.016682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:16.017808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:16.018608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 311/637] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:16.024174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0317f9b-940d-4b80-82cf-76a826a5dfa9 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:16.126802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0317f9b-940d-4b80-82cf-76a826a5dfa9 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:16.161096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef8a6a-f1b6-4181-8195-d13e7735deec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:16.162130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef8a6a-f1b6-4181-8195-d13e7735deec None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:16.162607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef8a6a-f1b6-4181-8195-d13e7735deec None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:16.163075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef8a6a-f1b6-4181-8195-d13e7735deec None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:16.163925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef8a6a-f1b6-4181-8195-d13e7735deec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:16.164480 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef8a6a-f1b6-4181-8195-d13e7735deec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:16.167481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef8a6a-f1b6-4181-8195-d13e7735deec None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:16.173408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 327/638] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:16.179104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4896a7-6332-4d4a-9a36-26ce1acb5b48 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:16.282250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4896a7-6332-4d4a-9a36-26ce1acb5b48 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:16.316819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688a358-450d-4d39-b3c8-e16963e7801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:16.317570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688a358-450d-4d39-b3c8-e16963e7801b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:16.317855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688a358-450d-4d39-b3c8-e16963e7801b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:16.318127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688a358-450d-4d39-b3c8-e16963e7801b None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:16.318753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688a358-450d-4d39-b3c8-e16963e7801b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:16.319057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688a358-450d-4d39-b3c8-e16963e7801b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:16.320604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688a358-450d-4d39-b3c8-e16963e7801b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:16.325689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 312/639] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:16.639600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fba62-d945-4d3f-a43f-df2bd728acbb None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:16.639942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fba62-d945-4d3f-a43f-df2bd728acbb None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:16.640149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fba62-d945-4d3f-a43f-df2bd728acbb None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:16.641764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 328/640] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:16.646832 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e15d4c-0743-4583-bb12-c86c8e3bea27 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:16.647264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e15d4c-0743-4583-bb12-c86c8e3bea27 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:16.647428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e15d4c-0743-4583-bb12-c86c8e3bea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:16.665453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-90e15d4c-0743-4583-bb12-c86c8e3bea27 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:16.667167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-90e15d4c-0743-4583-bb12-c86c8e3bea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:16.692448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:16.736400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 313/641] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:17.003070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36fba62-d945-4d3f-a43f-df2bd728acbb None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:17.015745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:17.111324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36fba62-d945-4d3f-a43f-df2bd728acbb None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:17.147041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a315e7b-86c5-4751-ae90-4b862df69be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:17.147672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a315e7b-86c5-4751-ae90-4b862df69be5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:17.147946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a315e7b-86c5-4751-ae90-4b862df69be5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:17.148217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a315e7b-86c5-4751-ae90-4b862df69be5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:17.148865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a315e7b-86c5-4751-ae90-4b862df69be5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:17.150226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a315e7b-86c5-4751-ae90-4b862df69be5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-0a315e7b-86c5-4751-ae90-4b862df69be5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:17.159533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:17.160578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:17.162260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 329/642] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:29:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:17.167115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e15d4c-0743-4583-bb12-c86c8e3bea27 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:17.269995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e15d4c-0743-4583-bb12-c86c8e3bea27 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:17.308175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1959a-3cac-461d-8e6c-7e8e6c2db1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:17.308843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1959a-3cac-461d-8e6c-7e8e6c2db1be None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:17.309126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1959a-3cac-461d-8e6c-7e8e6c2db1be None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:17.309401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1959a-3cac-461d-8e6c-7e8e6c2db1be None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:17.310053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1959a-3cac-461d-8e6c-7e8e6c2db1be None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:17.310360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1959a-3cac-461d-8e6c-7e8e6c2db1be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:17.311780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1959a-3cac-461d-8e6c-7e8e6c2db1be None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:17.317555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 314/643] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:29:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:17.325717 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a315e7b-86c5-4751-ae90-4b862df69be5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:17.404433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:17.429991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8f36-f8c1-492b-9c2d-175274def861 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:17.430355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8f36-f8c1-492b-9c2d-175274def861 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:17.430529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8f36-f8c1-492b-9c2d-175274def861 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:17.432542 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 315/644] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:29:17.436870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a315e7b-86c5-4751-ae90-4b862df69be5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:17.439511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f8fbc-09b1-4270-a084-a01a27fa3239 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:17.440473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f8fbc-09b1-4270-a084-a01a27fa3239 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:17.440473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f8fbc-09b1-4270-a084-a01a27fa3239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:17.460785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-027f8fbc-09b1-4270-a084-a01a27fa3239 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:17.462046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-027f8fbc-09b1-4270-a084-a01a27fa3239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:17.477767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6a043-5439-4351-99bb-dbc10de2b1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:17.479228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6a043-5439-4351-99bb-dbc10de2b1b6 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:17.479705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6a043-5439-4351-99bb-dbc10de2b1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:17.480172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6a043-5439-4351-99bb-dbc10de2b1b6 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:17.481132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6a043-5439-4351-99bb-dbc10de2b1b6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:17.481714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6a043-5439-4351-99bb-dbc10de2b1b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:17.483269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6a043-5439-4351-99bb-dbc10de2b1b6 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:17.484409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:17.489746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 330/645] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:29:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:17.540310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 316/646] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:17.548922 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270759-a094-4028-a857-d60a9de8d4d5 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:17.549209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270759-a094-4028-a857-d60a9de8d4d5 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:17.549486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270759-a094-4028-a857-d60a9de8d4d5 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:17.551099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 331/647] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:17.557822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ead6e0-dd3c-425e-8ecf-b340ca5c5c94 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:17.558432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ead6e0-dd3c-425e-8ecf-b340ca5c5c94 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:17.558900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ead6e0-dd3c-425e-8ecf-b340ca5c5c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:17.580100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-f2ead6e0-dd3c-425e-8ecf-b340ca5c5c94 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:17.581727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f2ead6e0-dd3c-425e-8ecf-b340ca5c5c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:17.660012 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 317/648] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:17.819540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67270759-a094-4028-a857-d60a9de8d4d5 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:17.922305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67270759-a094-4028-a857-d60a9de8d4d5 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:17.940217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ead6e0-dd3c-425e-8ecf-b340ca5c5c94 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:17.957049 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e137a93-e68a-4be2-9dc4-2b7d7ff597ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:17.957657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e137a93-e68a-4be2-9dc4-2b7d7ff597ab None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:17.957934 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e137a93-e68a-4be2-9dc4-2b7d7ff597ab None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:17.958206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e137a93-e68a-4be2-9dc4-2b7d7ff597ab None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:17.958859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e137a93-e68a-4be2-9dc4-2b7d7ff597ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:17.960230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e137a93-e68a-4be2-9dc4-2b7d7ff597ab None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:17.971356 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:17.977794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 332/649] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:29:17 2022] POST /identity/v3/services => generated 269 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:18.058837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:18.082261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ead6e0-dd3c-425e-8ecf-b340ca5c5c94 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:18.127884 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d079498-effc-496f-a63b-213e16cdf4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:18.128562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d079498-effc-496f-a63b-213e16cdf4b8 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:18.128846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d079498-effc-496f-a63b-213e16cdf4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:18.129122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d079498-effc-496f-a63b-213e16cdf4b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:18.141654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d079498-effc-496f-a63b-213e16cdf4b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-8d079498-effc-496f-a63b-213e16cdf4b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:18.146604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:18.147555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:18.148584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 318/650] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Oct 20 14:29:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:18.156379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e137a93-e68a-4be2-9dc4-2b7d7ff597ab None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:18.280910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e137a93-e68a-4be2-9dc4-2b7d7ff597ab None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:18.316001 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a009be6-d78b-47b9-a699-2ea259b37af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:18.316677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a009be6-d78b-47b9-a699-2ea259b37af8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:18.316957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a009be6-d78b-47b9-a699-2ea259b37af8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:18.317366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a009be6-d78b-47b9-a699-2ea259b37af8 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:18.318507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a009be6-d78b-47b9-a699-2ea259b37af8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:18.319017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a009be6-d78b-47b9-a699-2ea259b37af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:18.321301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a009be6-d78b-47b9-a699-2ea259b37af8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:18.331843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 333/651] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Oct 20 14:29:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:18.343105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d079498-effc-496f-a63b-213e16cdf4b8 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:18.459572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d079498-effc-496f-a63b-213e16cdf4b8 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:18.497632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992e52d-2147-4ef8-ae84-a8db98787748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:18.498619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992e52d-2147-4ef8-ae84-a8db98787748 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:18.499100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992e52d-2147-4ef8-ae84-a8db98787748 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:18.499585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992e52d-2147-4ef8-ae84-a8db98787748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:18.510817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992e52d-2147-4ef8-ae84-a8db98787748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:18.512590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992e52d-2147-4ef8-ae84-a8db98787748 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:18.514610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-f992e52d-2147-4ef8-ae84-a8db98787748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:18.531071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:18.532091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:18.532977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:18.532977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:18.532977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:18.534012 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 319/652] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:18.539733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a009be6-d78b-47b9-a699-2ea259b37af8 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:18.608388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:18.651944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a009be6-d78b-47b9-a699-2ea259b37af8 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:18.653351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04179ac5-2408-4f52-abc5-51eff1f80b6d None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:18.653644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04179ac5-2408-4f52-abc5-51eff1f80b6d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:18.653931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04179ac5-2408-4f52-abc5-51eff1f80b6d None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:18.655404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 320/653] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:18.660228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43dffb-8f35-43fd-a5af-862bc06a734a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:18.660796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43dffb-8f35-43fd-a5af-862bc06a734a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:18.660796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43dffb-8f35-43fd-a5af-862bc06a734a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:18.678915 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-cd43dffb-8f35-43fd-a5af-862bc06a734a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:18.680089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cd43dffb-8f35-43fd-a5af-862bc06a734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:18.690148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b690a48-b5c4-46c2-92c3-988b73152ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:18.691130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b690a48-b5c4-46c2-92c3-988b73152ebc None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:18.691707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b690a48-b5c4-46c2-92c3-988b73152ebc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:18.692333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b690a48-b5c4-46c2-92c3-988b73152ebc None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:18.693193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b690a48-b5c4-46c2-92c3-988b73152ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:18.693704 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b690a48-b5c4-46c2-92c3-988b73152ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:18.695558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b690a48-b5c4-46c2-92c3-988b73152ebc None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:18.710340 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 334/654] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:18.716482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b690a48-b5c4-46c2-92c3-988b73152ebc None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:18.765919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 321/655] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:18 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:18.828531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b690a48-b5c4-46c2-92c3-988b73152ebc None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:18.852913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:18.867098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d1634-e724-4f98-8cfb-8e7b733afca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:18.867900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d1634-e724-4f98-8cfb-8e7b733afca3 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:18.868174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d1634-e724-4f98-8cfb-8e7b733afca3 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:18.868471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d1634-e724-4f98-8cfb-8e7b733afca3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:18.875646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d1634-e724-4f98-8cfb-8e7b733afca3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:18.877101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d1634-e724-4f98-8cfb-8e7b733afca3 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-667d1634-e724-4f98-8cfb-8e7b733afca3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:18.888959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:18.889894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:18.890803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 335/656] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Oct 20 14:29:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:18.897177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43dffb-8f35-43fd-a5af-862bc06a734a None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:19.007907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43dffb-8f35-43fd-a5af-862bc06a734a None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:19.032630 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:19.048364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d1634-e724-4f98-8cfb-8e7b733afca3 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:19.048534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870343d3-ec20-43e4-b6d8-e713dd3608a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:19.049030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870343d3-ec20-43e4-b6d8-e713dd3608a9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:19.049313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870343d3-ec20-43e4-b6d8-e713dd3608a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:19.049588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870343d3-ec20-43e4-b6d8-e713dd3608a9 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:19.050320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870343d3-ec20-43e4-b6d8-e713dd3608a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:19.050629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870343d3-ec20-43e4-b6d8-e713dd3608a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:19.052541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870343d3-ec20-43e4-b6d8-e713dd3608a9 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:19.076316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 322/657] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Oct 20 14:29:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:19.082706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870343d3-ec20-43e4-b6d8-e713dd3608a9 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:19.149382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d1634-e724-4f98-8cfb-8e7b733afca3 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:19.186627 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8acd8-c145-438a-9d6b-3c044410a227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:19.187256 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8acd8-c145-438a-9d6b-3c044410a227 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:19.187617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8acd8-c145-438a-9d6b-3c044410a227 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:19.187955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8acd8-c145-438a-9d6b-3c044410a227 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:19.188857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8acd8-c145-438a-9d6b-3c044410a227 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:19.190328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8acd8-c145-438a-9d6b-3c044410a227 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:19.206086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870343d3-ec20-43e4-b6d8-e713dd3608a9 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:19.208233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 336/658] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:29:19 2022] POST /identity/v3/services => generated 241 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:29:19.262026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b00369-b91e-4708-a062-ef7f42b340a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:19.262675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b00369-b91e-4708-a062-ef7f42b340a4 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:19.262951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b00369-b91e-4708-a062-ef7f42b340a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:19.263227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b00369-b91e-4708-a062-ef7f42b340a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:19.276368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b00369-b91e-4708-a062-ef7f42b340a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:19.279215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b00369-b91e-4708-a062-ef7f42b340a4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-72b00369-b91e-4708-a062-ef7f42b340a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:19.296642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:19.297634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:19.298504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:19.298504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:19.298504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:19.298504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 323/659] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:19.305057 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8acd8-c145-438a-9d6b-3c044410a227 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:19.376738 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:19.422837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8acd8-c145-438a-9d6b-3c044410a227 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:19.458170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fec78-106b-4f9e-9b06-c0a705563817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:19.458795 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fec78-106b-4f9e-9b06-c0a705563817 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:19.459076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fec78-106b-4f9e-9b06-c0a705563817 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:19.459351 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fec78-106b-4f9e-9b06-c0a705563817 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:19.459998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fec78-106b-4f9e-9b06-c0a705563817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:19.460305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fec78-106b-4f9e-9b06-c0a705563817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:19.461743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fec78-106b-4f9e-9b06-c0a705563817 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:19.475033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 337/660] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:19.480678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b00369-b91e-4708-a062-ef7f42b340a4 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:19.516647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:19.589151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b00369-b91e-4708-a062-ef7f42b340a4 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:19.623054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b582f4-c330-452d-9687-e9e5f6d50a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:19.623684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b582f4-c330-452d-9687-e9e5f6d50a2c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:19.623965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b582f4-c330-452d-9687-e9e5f6d50a2c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:19.624238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b582f4-c330-452d-9687-e9e5f6d50a2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:19.630976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b582f4-c330-452d-9687-e9e5f6d50a2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:19.632461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b582f4-c330-452d-9687-e9e5f6d50a2c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-34b582f4-c330-452d-9687-e9e5f6d50a2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:19.644307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:19.645381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:19.646467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:19.646467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:19.646467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:19.646467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:19.646467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:19.646467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:19.646467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:19.646467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:19.646467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 324/661] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:19.652926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fec78-106b-4f9e-9b06-c0a705563817 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:19.761226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:19.767617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fec78-106b-4f9e-9b06-c0a705563817 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:19.800903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949b9fe-acf5-48ec-8502-0556fbbc940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:19.801505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949b9fe-acf5-48ec-8502-0556fbbc940a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:19.801777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949b9fe-acf5-48ec-8502-0556fbbc940a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:19.802048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949b9fe-acf5-48ec-8502-0556fbbc940a None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:19.802730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949b9fe-acf5-48ec-8502-0556fbbc940a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:19.803031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949b9fe-acf5-48ec-8502-0556fbbc940a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:19.804478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949b9fe-acf5-48ec-8502-0556fbbc940a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:19.816558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 338/662] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:19.822949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b582f4-c330-452d-9687-e9e5f6d50a2c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:19.854307 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3cd14-24f5-4c6c-b454-e171743ef554 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:19.854647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3cd14-24f5-4c6c-b454-e171743ef554 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:19.854907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3cd14-24f5-4c6c-b454-e171743ef554 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:19.856876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 339/663] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:29:19.862429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c16f81-68ab-4a46-afe9-d293d9ea9a21 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:19.862718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c16f81-68ab-4a46-afe9-d293d9ea9a21 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:19.863000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c16f81-68ab-4a46-afe9-d293d9ea9a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:19.883464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-27c16f81-68ab-4a46-afe9-d293d9ea9a21 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:19.885479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-27c16f81-68ab-4a46-afe9-d293d9ea9a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:19.926766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:19.939420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b582f4-c330-452d-9687-e9e5f6d50a2c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:19.969035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 340/664] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:19 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:19.982839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2080c7-223f-4a92-b4e1-127062f0982a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:19.983495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2080c7-223f-4a92-b4e1-127062f0982a None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:19.983770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2080c7-223f-4a92-b4e1-127062f0982a None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:19.984045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2080c7-223f-4a92-b4e1-127062f0982a None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/04f97b428a6347e39b9b1f97ace1e93b/roles/1430408cb7654b4889c5df352e3c9b26` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:19.986186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2080c7-223f-4a92-b4e1-127062f0982a None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=04f97b428a6347e39b9b1f97ace1e93b, role_id=1430408cb7654b4889c5df352e3c9b26)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:19.987631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2080c7-223f-4a92-b4e1-127062f0982a None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:20.004223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 325/665] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:29:19 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/04f97b428a6347e39b9b1f97ace1e93b/roles/1430408cb7654b4889c5df352e3c9b26 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:29:20.258249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c16f81-68ab-4a46-afe9-d293d9ea9a21 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:20.326948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a25cf1-b73a-4f66-be9f-2c286ad6f003 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:20.327277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a25cf1-b73a-4f66-be9f-2c286ad6f003 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:20.327510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a25cf1-b73a-4f66-be9f-2c286ad6f003 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:20.329158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 326/666] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:20.334282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22b2dd-a9f6-4aae-a00e-88f3ec602861 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:20.334568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22b2dd-a9f6-4aae-a00e-88f3ec602861 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:20.334846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22b2dd-a9f6-4aae-a00e-88f3ec602861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:20.356701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:20.362881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c16f81-68ab-4a46-afe9-d293d9ea9a21 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:20.366425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-2e22b2dd-a9f6-4aae-a00e-88f3ec602861 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:20.368113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2e22b2dd-a9f6-4aae-a00e-88f3ec602861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:20.401177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612787ad-00e0-4a25-8fbe-ee98c1ef0900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:20.402223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612787ad-00e0-4a25-8fbe-ee98c1ef0900 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:20.402723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612787ad-00e0-4a25-8fbe-ee98c1ef0900 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:20.403226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612787ad-00e0-4a25-8fbe-ee98c1ef0900 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:20.404089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612787ad-00e0-4a25-8fbe-ee98c1ef0900 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:20.405676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612787ad-00e0-4a25-8fbe-ee98c1ef0900 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:20.424235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 341/667] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:29:20 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:20.463860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 327/668] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:20.746960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612787ad-00e0-4a25-8fbe-ee98c1ef0900 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:20.854109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612787ad-00e0-4a25-8fbe-ee98c1ef0900 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:20.891071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6520e66-b1c8-40f0-98dc-526a0858053a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:20.892116 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6520e66-b1c8-40f0-98dc-526a0858053a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:20.892696 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6520e66-b1c8-40f0-98dc-526a0858053a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:20.893184 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6520e66-b1c8-40f0-98dc-526a0858053a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:20.894025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6520e66-b1c8-40f0-98dc-526a0858053a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:20.895669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6520e66-b1c8-40f0-98dc-526a0858053a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-f6520e66-b1c8-40f0-98dc-526a0858053a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:20.906756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:20.907766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 14:29:20.908732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:20.916534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 342/669] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Oct 20 14:29:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:20.918311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22b2dd-a9f6-4aae-a00e-88f3ec602861 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:21.002214 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:21.028037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22b2dd-a9f6-4aae-a00e-88f3ec602861 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:21.063625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91483766-3848-4774-8140-91424463c6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:21.064660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91483766-3848-4774-8140-91424463c6a0 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:21.065143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91483766-3848-4774-8140-91424463c6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:21.065651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91483766-3848-4774-8140-91424463c6a0 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:21.066511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91483766-3848-4774-8140-91424463c6a0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:21.067013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91483766-3848-4774-8140-91424463c6a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:21.068614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91483766-3848-4774-8140-91424463c6a0 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:21.078673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 328/670] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:21.080203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6520e66-b1c8-40f0-98dc-526a0858053a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:21.100757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:21.190612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6520e66-b1c8-40f0-98dc-526a0858053a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:21.227846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89a22d-9864-46dc-8ae3-09c17137dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:21.229568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89a22d-9864-46dc-8ae3-09c17137dedc None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:21.229858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89a22d-9864-46dc-8ae3-09c17137dedc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:21.230134 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89a22d-9864-46dc-8ae3-09c17137dedc None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:21.230770 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89a22d-9864-46dc-8ae3-09c17137dedc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:21.231075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89a22d-9864-46dc-8ae3-09c17137dedc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:21.232464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89a22d-9864-46dc-8ae3-09c17137dedc None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:21.242895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 343/671] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:21.245078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91483766-3848-4774-8140-91424463c6a0 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:21.360284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91483766-3848-4774-8140-91424463c6a0 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:21.398124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4653193-1a7f-4c2a-b0a5-9c911a823214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:21.398733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4653193-1a7f-4c2a-b0a5-9c911a823214 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:21.399011 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4653193-1a7f-4c2a-b0a5-9c911a823214 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:21.399283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4653193-1a7f-4c2a-b0a5-9c911a823214 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:21.399978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4653193-1a7f-4c2a-b0a5-9c911a823214 None admin] RBAC: Adding query filter params (interface=public, service_id=5a882704c99c4e608ee20633507e059e, region_id=RegionOne) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:21.400877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4653193-1a7f-4c2a-b0a5-9c911a823214 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:21.402266 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4653193-1a7f-4c2a-b0a5-9c911a823214 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:21.411957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 329/672] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Oct 20 14:29:21 2022] GET /identity/v3/endpoints?service_id=5a882704c99c4e608ee20633507e059e&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:21.413403 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89a22d-9864-46dc-8ae3-09c17137dedc None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:21.516826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:21.523662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89a22d-9864-46dc-8ae3-09c17137dedc None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:21.561813 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae8a88-742b-4653-88a9-a68ed89caa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:21.562421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8a88-742b-4653-88a9-a68ed89caa25 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:21.562700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8a88-742b-4653-88a9-a68ed89caa25 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:21.562973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8a88-742b-4653-88a9-a68ed89caa25 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:21.563569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae8a88-742b-4653-88a9-a68ed89caa25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:21.564975 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae8a88-742b-4653-88a9-a68ed89caa25 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:21.576386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 344/673] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Oct 20 14:29:21 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:29:21.608696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bc324-629a-4386-8da2-b7c056832225 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:21.608889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bc324-629a-4386-8da2-b7c056832225 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:21.609166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bc324-629a-4386-8da2-b7c056832225 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:21.610953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 330/674] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:21.616364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be32fc-825f-46a1-8652-9edf240452c2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:21.616654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be32fc-825f-46a1-8652-9edf240452c2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:21.616936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be32fc-825f-46a1-8652-9edf240452c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:21.636140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e3be32fc-825f-46a1-8652-9edf240452c2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:21.637398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e3be32fc-825f-46a1-8652-9edf240452c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:21.710797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 345/675] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:21.984855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bc324-629a-4386-8da2-b7c056832225 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:21.988778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:22.097405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bc324-629a-4386-8da2-b7c056832225 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:22.135747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbab76-3526-47d2-bc77-7e3f7eac51d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:22.136397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbab76-3526-47d2-bc77-7e3f7eac51d9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.136678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbab76-3526-47d2-bc77-7e3f7eac51d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.136950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbab76-3526-47d2-bc77-7e3f7eac51d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.137525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbab76-3526-47d2-bc77-7e3f7eac51d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:22.138889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbab76-3526-47d2-bc77-7e3f7eac51d9 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-89bbab76-3526-47d2-bc77-7e3f7eac51d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:22.147997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:22.148994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:22.149814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 331/676] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:22.155441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be32fc-825f-46a1-8652-9edf240452c2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:22.198788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c07a1-6a64-4a1b-b506-37411bc45e71 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.199134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c07a1-6a64-4a1b-b506-37411bc45e71 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.199329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c07a1-6a64-4a1b-b506-37411bc45e71 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.200673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 332/677] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:29:22.205586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078425b-223d-4527-a346-16b0e08ed780 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.205864 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078425b-223d-4527-a346-16b0e08ed780 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.206138 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078425b-223d-4527-a346-16b0e08ed780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.227982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-4078425b-223d-4527-a346-16b0e08ed780 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:22.229221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:22.229541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4078425b-223d-4527-a346-16b0e08ed780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:22.268447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be32fc-825f-46a1-8652-9edf240452c2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:22.305729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:22.313352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 333/678] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:22.317128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027870e-3151-4d85-a869-5d9e5c44b8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:22.317901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027870e-3151-4d85-a869-5d9e5c44b8e8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.318276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027870e-3151-4d85-a869-5d9e5c44b8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.318639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027870e-3151-4d85-a869-5d9e5c44b8e8 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.319369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027870e-3151-4d85-a869-5d9e5c44b8e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:22.319784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027870e-3151-4d85-a869-5d9e5c44b8e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:22.321349 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027870e-3151-4d85-a869-5d9e5c44b8e8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:22.326773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 346/679] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:22.332436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078425b-223d-4527-a346-16b0e08ed780 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:22.439077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078425b-223d-4527-a346-16b0e08ed780 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:22.474811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02416-7ea5-405b-a495-10aeadf49c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:22.475411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02416-7ea5-405b-a495-10aeadf49c64 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.475686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02416-7ea5-405b-a495-10aeadf49c64 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.475954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02416-7ea5-405b-a495-10aeadf49c64 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.476597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02416-7ea5-405b-a495-10aeadf49c64 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:22.476948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02416-7ea5-405b-a495-10aeadf49c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:22.478434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02416-7ea5-405b-a495-10aeadf49c64 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:22.483593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 334/680] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:22.489458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027870e-3151-4d85-a869-5d9e5c44b8e8 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:22.595622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027870e-3151-4d85-a869-5d9e5c44b8e8 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:22.615220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02416-7ea5-405b-a495-10aeadf49c64 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:22.642262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e258b8-96a0-487f-8267-198bc82fd958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:22.642871 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e258b8-96a0-487f-8267-198bc82fd958 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.643146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e258b8-96a0-487f-8267-198bc82fd958 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.643422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e258b8-96a0-487f-8267-198bc82fd958 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.644117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e258b8-96a0-487f-8267-198bc82fd958 None admin] RBAC: Adding query filter params (interface=public, service_id=016568d932fe494f83773c97dfbcb6bc, region_id=RegionOne) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:22.644463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e258b8-96a0-487f-8267-198bc82fd958 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:22.646378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e258b8-96a0-487f-8267-198bc82fd958 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:22.649805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:22.655091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 347/681] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Oct 20 14:29:22 2022] GET /identity/v3/endpoints?service_id=016568d932fe494f83773c97dfbcb6bc&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:22.661464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e258b8-96a0-487f-8267-198bc82fd958 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:22.706379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:22.750289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02416-7ea5-405b-a495-10aeadf49c64 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:22.782280 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e258b8-96a0-487f-8267-198bc82fd958 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:22.784805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1938b8-e2e3-4c80-8967-f00086238f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:22.785429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1938b8-e2e3-4c80-8967-f00086238f86 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.785710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1938b8-e2e3-4c80-8967-f00086238f86 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.785979 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1938b8-e2e3-4c80-8967-f00086238f86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.796524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1938b8-e2e3-4c80-8967-f00086238f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-3b1938b8-e2e3-4c80-8967-f00086238f86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:22.801533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:22.802549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 14:29:22.803582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:22.805522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 335/682] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Oct 20 14:29:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:22.808497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe139c-1206-4764-8a41-fe72113f36a1 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.808497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe139c-1206-4764-8a41-fe72113f36a1 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.808497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe139c-1206-4764-8a41-fe72113f36a1 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.810767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 336/683] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:22.811661 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe139c-1206-4764-8a41-fe72113f36a1 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:22.826416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9437e51-bcbd-4646-b9b2-c35e04490d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:22.827027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9437e51-bcbd-4646-b9b2-c35e04490d2d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.827310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9437e51-bcbd-4646-b9b2-c35e04490d2d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.827588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9437e51-bcbd-4646-b9b2-c35e04490d2d None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.828188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9437e51-bcbd-4646-b9b2-c35e04490d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:22.829626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9437e51-bcbd-4646-b9b2-c35e04490d2d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:22.836311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 348/684] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Oct 20 14:29:22 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:29:22.837406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a32c7-b1a4-4a47-9710-88ff8070ce8f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.837866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a32c7-b1a4-4a47-9710-88ff8070ce8f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.838153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a32c7-b1a4-4a47-9710-88ff8070ce8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.858290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-b77a32c7-b1a4-4a47-9710-88ff8070ce8f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:22.859624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b77a32c7-b1a4-4a47-9710-88ff8070ce8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:22.928186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:22.938348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe139c-1206-4764-8a41-fe72113f36a1 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:22.944825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 349/685] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:22.966939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:22.981420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6c143-840d-4efd-8a78-bdb3b86f7c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:22.982402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6c143-840d-4efd-8a78-bdb3b86f7c62 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:22.982958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6c143-840d-4efd-8a78-bdb3b86f7c62 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:22.983424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6c143-840d-4efd-8a78-bdb3b86f7c62 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:22.984355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6c143-840d-4efd-8a78-bdb3b86f7c62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:22.984859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6c143-840d-4efd-8a78-bdb3b86f7c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:22.986441 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6c143-840d-4efd-8a78-bdb3b86f7c62 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:22.998172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 337/686] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Oct 20 14:29:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:23.004457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a32c7-b1a4-4a47-9710-88ff8070ce8f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.114533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a32c7-b1a4-4a47-9710-88ff8070ce8f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:23.149554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4d314-b47e-41c5-befe-6070526cb9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:23.150711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4d314-b47e-41c5-befe-6070526cb9f1 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.150989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4d314-b47e-41c5-befe-6070526cb9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.151263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4d314-b47e-41c5-befe-6070526cb9f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.161934 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4d314-b47e-41c5-befe-6070526cb9f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:23.163461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4d314-b47e-41c5-befe-6070526cb9f1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-78c4d314-b47e-41c5-befe-6070526cb9f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:23.179023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:23.179955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:23.180854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:23.180854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:23.180854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:23.180854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 350/687] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:23.187309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6c143-840d-4efd-8a78-bdb3b86f7c62 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.239063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4d314-b47e-41c5-befe-6070526cb9f1 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.289425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6c143-840d-4efd-8a78-bdb3b86f7c62 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:23.293477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:23.325334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5402d-8219-4008-93e6-3d59fea9007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:23.326299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5402d-8219-4008-93e6-3d59fea9007a None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.326773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5402d-8219-4008-93e6-3d59fea9007a None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.327265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5402d-8219-4008-93e6-3d59fea9007a None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.328189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5402d-8219-4008-93e6-3d59fea9007a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:23.328703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5402d-8219-4008-93e6-3d59fea9007a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:23.330268 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5402d-8219-4008-93e6-3d59fea9007a None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:23.343917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 338/688] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:23.349912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5402d-8219-4008-93e6-3d59fea9007a None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.352548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4d314-b47e-41c5-befe-6070526cb9f1 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:23.393058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1760083-427f-4649-81b5-933a5451abcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:23.393688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1760083-427f-4649-81b5-933a5451abcc None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.393961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1760083-427f-4649-81b5-933a5451abcc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.394229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1760083-427f-4649-81b5-933a5451abcc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.394861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1760083-427f-4649-81b5-933a5451abcc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:23.396271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1760083-427f-4649-81b5-933a5451abcc None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-c1760083-427f-4649-81b5-933a5451abcc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:23.407713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:23.408994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 14:29:23.409804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:23.412415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 351/689] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:23.414588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1760083-427f-4649-81b5-933a5451abcc None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.440758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:23.481606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5402d-8219-4008-93e6-3d59fea9007a None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:23.531721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1760083-427f-4649-81b5-933a5451abcc None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:23.536589 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a0241-06c6-4e6a-9d9d-f3b284ff8fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:23.537232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a0241-06c6-4e6a-9d9d-f3b284ff8fe0 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.537515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a0241-06c6-4e6a-9d9d-f3b284ff8fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.537808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a0241-06c6-4e6a-9d9d-f3b284ff8fe0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.546421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:23.547922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a0241-06c6-4e6a-9d9d-f3b284ff8fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:23.549579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a0241-06c6-4e6a-9d9d-f3b284ff8fe0 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-ad0a0241-06c6-4e6a-9d9d-f3b284ff8fe0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:23.566821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:23.567867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:23.568807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 339/690] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:23.574349 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a0241-06c6-4e6a-9d9d-f3b284ff8fe0 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.576020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23437a79-fdb9-4153-8ff0-03d3a9f5803a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:23.576854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23437a79-fdb9-4153-8ff0-03d3a9f5803a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.577308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23437a79-fdb9-4153-8ff0-03d3a9f5803a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.577743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23437a79-fdb9-4153-8ff0-03d3a9f5803a None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.578522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23437a79-fdb9-4153-8ff0-03d3a9f5803a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:23.578822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23437a79-fdb9-4153-8ff0-03d3a9f5803a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:23.580191 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23437a79-fdb9-4153-8ff0-03d3a9f5803a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:23.590407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 352/691] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:23.591495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23437a79-fdb9-4153-8ff0-03d3a9f5803a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.696803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a0241-06c6-4e6a-9d9d-f3b284ff8fe0 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:23.697670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23437a79-fdb9-4153-8ff0-03d3a9f5803a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:23.700037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:23.733252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2f082-c74a-40c9-8cc7-33c0dd9eee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:23.733900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2f082-c74a-40c9-8cc7-33c0dd9eee80 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.734134 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2f082-c74a-40c9-8cc7-33c0dd9eee80 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.734407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2f082-c74a-40c9-8cc7-33c0dd9eee80 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.735027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2f082-c74a-40c9-8cc7-33c0dd9eee80 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:23.735330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2f082-c74a-40c9-8cc7-33c0dd9eee80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:23.736793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2f082-c74a-40c9-8cc7-33c0dd9eee80 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:23.738769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42d737-9da1-4893-9909-9ef477aa957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:23.739617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42d737-9da1-4893-9909-9ef477aa957d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.740098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42d737-9da1-4893-9909-9ef477aa957d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.740629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42d737-9da1-4893-9909-9ef477aa957d None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.741574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42d737-9da1-4893-9909-9ef477aa957d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:23.742084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42d737-9da1-4893-9909-9ef477aa957d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:23.744564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42d737-9da1-4893-9909-9ef477aa957d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:23.747338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 353/692] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:23.748639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2f082-c74a-40c9-8cc7-33c0dd9eee80 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.765963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 340/693] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:23.776806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42d737-9da1-4893-9909-9ef477aa957d None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.800774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:23.855771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2f082-c74a-40c9-8cc7-33c0dd9eee80 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:23.885831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42d737-9da1-4893-9909-9ef477aa957d None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:23.890840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62cc12-9c51-4aa5-afd1-6ee307a17982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:23.891447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62cc12-9c51-4aa5-afd1-6ee307a17982 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.891724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62cc12-9c51-4aa5-afd1-6ee307a17982 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.891994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62cc12-9c51-4aa5-afd1-6ee307a17982 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.892717 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62cc12-9c51-4aa5-afd1-6ee307a17982 None admin] RBAC: Adding query filter params (interface=public, service_id=960b04c58a734d3f898e6baef6d820d0, region_id=RegionOne) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:23.893023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62cc12-9c51-4aa5-afd1-6ee307a17982 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:23.894387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62cc12-9c51-4aa5-afd1-6ee307a17982 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:23.899320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 354/694] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/endpoints?service_id=960b04c58a734d3f898e6baef6d820d0&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:23.901030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b335ead-74c1-40bd-a8a6-f249b364b28b None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.901301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b335ead-74c1-40bd-a8a6-f249b364b28b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.901572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b335ead-74c1-40bd-a8a6-f249b364b28b None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.903078 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 355/695] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:23.908090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b335ead-74c1-40bd-a8a6-f249b364b28b None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:23.922800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e4594-1d0f-4087-9dc0-fd994f30190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:23.923438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e4594-1d0f-4087-9dc0-fd994f30190b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:23.923719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e4594-1d0f-4087-9dc0-fd994f30190b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:23.924000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e4594-1d0f-4087-9dc0-fd994f30190b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:23.956188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:23.995557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e4594-1d0f-4087-9dc0-fd994f30190b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:23.997027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e4594-1d0f-4087-9dc0-fd994f30190b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:24.021535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b335ead-74c1-40bd-a8a6-f249b364b28b None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:24.023581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:24.062293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55ac02-4ccc-4620-b883-fb28a781937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:24.062900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55ac02-4ccc-4620-b883-fb28a781937d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:24.063174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55ac02-4ccc-4620-b883-fb28a781937d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:24.063440 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55ac02-4ccc-4620-b883-fb28a781937d None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:24.064043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55ac02-4ccc-4620-b883-fb28a781937d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:24.065625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55ac02-4ccc-4620-b883-fb28a781937d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:24.074007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 356/696] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:29:24.075531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d005b-5bd2-42e7-ba21-6acd088e365f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:24.075833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d005b-5bd2-42e7-ba21-6acd088e365f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:24.076090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d005b-5bd2-42e7-ba21-6acd088e365f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:24.095944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e50d005b-5bd2-42e7-ba21-6acd088e365f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:24.097233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e50d005b-5bd2-42e7-ba21-6acd088e365f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:24.120392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 341/697] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:29:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 14:29:24.126115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e4594-1d0f-4087-9dc0-fd994f30190b None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:24.184222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 357/698] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:24.209184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:24.253620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e4594-1d0f-4087-9dc0-fd994f30190b None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:24.289524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4919007-8b1d-4575-b940-d0dd1c893f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:24.290244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4919007-8b1d-4575-b940-d0dd1c893f9c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:24.290528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4919007-8b1d-4575-b940-d0dd1c893f9c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:24.291340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4919007-8b1d-4575-b940-d0dd1c893f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:24.301574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4919007-8b1d-4575-b940-d0dd1c893f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:24.303166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4919007-8b1d-4575-b940-d0dd1c893f9c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-b4919007-8b1d-4575-b940-d0dd1c893f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:24.318715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:24.319689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:24.320687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:24.320687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:24.320687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:24.320687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 342/699] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:24.326021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d005b-5bd2-42e7-ba21-6acd088e365f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:24.436413 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:24.442398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d005b-5bd2-42e7-ba21-6acd088e365f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:24.477404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6095c0-2ca8-4283-9e3a-6513fce2fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:24.477932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6095c0-2ca8-4283-9e3a-6513fce2fd02 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:24.478344 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6095c0-2ca8-4283-9e3a-6513fce2fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:24.478636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6095c0-2ca8-4283-9e3a-6513fce2fd02 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:24.479389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6095c0-2ca8-4283-9e3a-6513fce2fd02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:24.479694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6095c0-2ca8-4283-9e3a-6513fce2fd02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:24.481100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6095c0-2ca8-4283-9e3a-6513fce2fd02 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:24.489710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4919007-8b1d-4575-b940-d0dd1c893f9c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:24.496385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 358/700] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:24.499448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6095c0-2ca8-4283-9e3a-6513fce2fd02 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:24.611086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4919007-8b1d-4575-b940-d0dd1c893f9c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:24.612418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6095c0-2ca8-4283-9e3a-6513fce2fd02 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:24.630676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:24.655859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea79af-3b90-4ef4-8fe1-fc6b7c25cb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:24.656526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea79af-3b90-4ef4-8fe1-fc6b7c25cb6b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:24.656808 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea79af-3b90-4ef4-8fe1-fc6b7c25cb6b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:24.657087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea79af-3b90-4ef4-8fe1-fc6b7c25cb6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:24.660299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6b5cd-7325-4ba2-a7d9-e59629d32dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:24.661281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6b5cd-7325-4ba2-a7d9-e59629d32dd5 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:24.661762 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6b5cd-7325-4ba2-a7d9-e59629d32dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:24.662248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6b5cd-7325-4ba2-a7d9-e59629d32dd5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:24.663181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6b5cd-7325-4ba2-a7d9-e59629d32dd5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:24.664065 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea79af-3b90-4ef4-8fe1-fc6b7c25cb6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:24.665056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6b5cd-7325-4ba2-a7d9-e59629d32dd5 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:24.666335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea79af-3b90-4ef4-8fe1-fc6b7c25cb6b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-93c6b5cd-7325-4ba2-a7d9-e59629d32dd5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:24.676559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:24.677515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 14:29:24.678457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-a9ea79af-3b90-4ef4-8fe1-fc6b7c25cb6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:24.679157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:24.679920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:24.682336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:24.682336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:24.682336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:24.682336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:24.682336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:24.682336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:24.682336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:24.682336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:24.682336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 359/701] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:24.683455 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 343/702] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Oct 20 14:29:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:24.684768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea79af-3b90-4ef4-8fe1-fc6b7c25cb6b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:24.689371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6b5cd-7325-4ba2-a7d9-e59629d32dd5 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:24.708017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:24.805780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6b5cd-7325-4ba2-a7d9-e59629d32dd5 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:24.810128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea79af-3b90-4ef4-8fe1-fc6b7c25cb6b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:24.842810 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47179974-94a5-4bba-ac6a-fd023f705a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:24.843804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47179974-94a5-4bba-ac6a-fd023f705a6c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:24.844312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47179974-94a5-4bba-ac6a-fd023f705a6c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:24.844790 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47179974-94a5-4bba-ac6a-fd023f705a6c None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:24.845594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47179974-94a5-4bba-ac6a-fd023f705a6c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:24.846193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47179974-94a5-4bba-ac6a-fd023f705a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:24.847861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47179974-94a5-4bba-ac6a-fd023f705a6c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:24.852477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85816f-ca55-4375-b5ba-afc69e0d784e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:24.853291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85816f-ca55-4375-b5ba-afc69e0d784e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:24.853842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85816f-ca55-4375-b5ba-afc69e0d784e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:24.854116 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85816f-ca55-4375-b5ba-afc69e0d784e None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:24.854800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85816f-ca55-4375-b5ba-afc69e0d784e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:24.854987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 344/703] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:24.855103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85816f-ca55-4375-b5ba-afc69e0d784e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:24.860774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47179974-94a5-4bba-ac6a-fd023f705a6c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:24.861149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85816f-ca55-4375-b5ba-afc69e0d784e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:24.880502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 360/704] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:24.883036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85816f-ca55-4375-b5ba-afc69e0d784e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:24.913893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:24.982164 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47179974-94a5-4bba-ac6a-fd023f705a6c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:24.986590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:25.013099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85816f-ca55-4375-b5ba-afc69e0d784e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:25.034080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825c661-f0b8-4b3d-9463-370d8dcdb735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:25.035795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825c661-f0b8-4b3d-9463-370d8dcdb735 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:25.036230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825c661-f0b8-4b3d-9463-370d8dcdb735 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:25.036684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825c661-f0b8-4b3d-9463-370d8dcdb735 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:25.037527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825c661-f0b8-4b3d-9463-370d8dcdb735 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:25.037983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825c661-f0b8-4b3d-9463-370d8dcdb735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:25.041191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825c661-f0b8-4b3d-9463-370d8dcdb735 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:25.053934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 345/705] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:25.056056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825c661-f0b8-4b3d-9463-370d8dcdb735 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:25.062002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf9fb0-a81e-4ea9-87ed-2a194110ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:25.062427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf9fb0-a81e-4ea9-87ed-2a194110ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:25.062706 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf9fb0-a81e-4ea9-87ed-2a194110ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:25.063033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf9fb0-a81e-4ea9-87ed-2a194110ebd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:25.063769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf9fb0-a81e-4ea9-87ed-2a194110ebd2 None admin] RBAC: Adding query filter params (role.id=1430408cb7654b4889c5df352e3c9b26, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=04f97b428a6347e39b9b1f97ace1e93b) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:25.064124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf9fb0-a81e-4ea9-87ed-2a194110ebd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:25.065683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf9fb0-a81e-4ea9-87ed-2a194110ebd2 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:25.080384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 361/706] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:29:24 2022] GET /identity/v3/role_assignments?user.id=04f97b428a6347e39b9b1f97ace1e93b&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=1430408cb7654b4889c5df352e3c9b26 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:25.153139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61471e83-8401-461d-8dbb-a0048c244f05 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:25.153588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61471e83-8401-461d-8dbb-a0048c244f05 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:25.153935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61471e83-8401-461d-8dbb-a0048c244f05 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:25.155913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 362/707] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:29:25.162146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aecdbd-1b2d-4317-873f-21f595d7e7a5 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:25.162569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aecdbd-1b2d-4317-873f-21f595d7e7a5 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:25.162951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aecdbd-1b2d-4317-873f-21f595d7e7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:25.183662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-97aecdbd-1b2d-4317-873f-21f595d7e7a5 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:25.185120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-97aecdbd-1b2d-4317-873f-21f595d7e7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:25.186963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825c661-f0b8-4b3d-9463-370d8dcdb735 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:25.195095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:25.232587 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194a1d5-faea-4ddf-9757-55753353367e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:25.233527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a1d5-faea-4ddf-9757-55753353367e None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:25.233811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a1d5-faea-4ddf-9757-55753353367e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:25.234086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a1d5-faea-4ddf-9757-55753353367e None admin] PATH_INFO: `/v3/endpoints` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:25.234803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194a1d5-faea-4ddf-9757-55753353367e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:25.237343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194a1d5-faea-4ddf-9757-55753353367e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:25.268437 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 346/708] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:29:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:25.296143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 363/709] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:25.570596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194a1d5-faea-4ddf-9757-55753353367e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:25.676101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194a1d5-faea-4ddf-9757-55753353367e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:25.712208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ff5dc-e828-4c39-b56f-b0dab03a1336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:25.713429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ff5dc-e828-4c39-b56f-b0dab03a1336 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:25.713914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ff5dc-e828-4c39-b56f-b0dab03a1336 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:25.714418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ff5dc-e828-4c39-b56f-b0dab03a1336 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:25.715202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ff5dc-e828-4c39-b56f-b0dab03a1336 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:25.716824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ff5dc-e828-4c39-b56f-b0dab03a1336 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-398ff5dc-e828-4c39-b56f-b0dab03a1336 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:25.726113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:25.727078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:25.727886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:25.731069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 347/710] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:25.737225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aecdbd-1b2d-4317-873f-21f595d7e7a5 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:25.805549 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:25.845276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aecdbd-1b2d-4317-873f-21f595d7e7a5 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:25.881141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3deaf0b-60fb-41aa-9048-802414f51947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:25.881801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deaf0b-60fb-41aa-9048-802414f51947 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:25.882084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deaf0b-60fb-41aa-9048-802414f51947 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:25.882358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deaf0b-60fb-41aa-9048-802414f51947 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:25.882990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3deaf0b-60fb-41aa-9048-802414f51947 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:25.883291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3deaf0b-60fb-41aa-9048-802414f51947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:25.885609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3deaf0b-60fb-41aa-9048-802414f51947 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:25.890994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 364/711] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:25.896735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ff5dc-e828-4c39-b56f-b0dab03a1336 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:25.927487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:26.007909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ff5dc-e828-4c39-b56f-b0dab03a1336 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:26.045856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2382d5-5aa9-45df-b853-37b771f163f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:26.046491 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2382d5-5aa9-45df-b853-37b771f163f5 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:26.046746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2382d5-5aa9-45df-b853-37b771f163f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:26.047024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2382d5-5aa9-45df-b853-37b771f163f5 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:26.047737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2382d5-5aa9-45df-b853-37b771f163f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:26.048047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2382d5-5aa9-45df-b853-37b771f163f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:26.049570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2382d5-5aa9-45df-b853-37b771f163f5 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:26.055800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 348/712] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:26.062121 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3deaf0b-60fb-41aa-9048-802414f51947 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:26.166745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3deaf0b-60fb-41aa-9048-802414f51947 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:26.200324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:26.203888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0135c4-e840-41a6-993a-a1ab159f2216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:26.204614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0135c4-e840-41a6-993a-a1ab159f2216 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:26.204994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0135c4-e840-41a6-993a-a1ab159f2216 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:26.205268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0135c4-e840-41a6-993a-a1ab159f2216 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:26.205966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0135c4-e840-41a6-993a-a1ab159f2216 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:26.207459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0135c4-e840-41a6-993a-a1ab159f2216 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:26.233219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 365/713] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:29:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:26.351833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7029-a1be-4f66-b561-0347a22bcba2 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:26.352168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7029-a1be-4f66-b561-0347a22bcba2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:26.352451 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7029-a1be-4f66-b561-0347a22bcba2 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:26.355633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 349/714] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:26.361762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5c27-5ba4-48dc-ae50-21693f8e59b0 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:26.362057 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5c27-5ba4-48dc-ae50-21693f8e59b0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:26.362311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5c27-5ba4-48dc-ae50-21693f8e59b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:26.382210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1a6f5c27-5ba4-48dc-ae50-21693f8e59b0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:26.383406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1a6f5c27-5ba4-48dc-ae50-21693f8e59b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:26.461400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 366/715] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:26.728641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7029-a1be-4f66-b561-0347a22bcba2 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:26.824011 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:26.835034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7029-a1be-4f66-b561-0347a22bcba2 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:26.874002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238af1-e3fa-4648-b523-0c70346f09c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:26.874629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238af1-e3fa-4648-b523-0c70346f09c1 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:26.874906 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238af1-e3fa-4648-b523-0c70346f09c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:26.875180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238af1-e3fa-4648-b523-0c70346f09c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:26.875757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a238af1-e3fa-4648-b523-0c70346f09c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:26.877169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a238af1-e3fa-4648-b523-0c70346f09c1 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-0a238af1-e3fa-4648-b523-0c70346f09c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:26.886274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:26.888709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:26.890680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 350/716] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:29:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:26.896949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f5c27-5ba4-48dc-ae50-21693f8e59b0 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:27.001801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f5c27-5ba4-48dc-ae50-21693f8e59b0 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:27.015876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:27.039162 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a82dd-a072-4178-b7a4-acbf473e3ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:27.039945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a82dd-a072-4178-b7a4-acbf473e3ead None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:27.040460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a82dd-a072-4178-b7a4-acbf473e3ead None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:27.040976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a82dd-a072-4178-b7a4-acbf473e3ead None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:27.041619 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a82dd-a072-4178-b7a4-acbf473e3ead None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:27.041924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a82dd-a072-4178-b7a4-acbf473e3ead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:27.043298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a82dd-a072-4178-b7a4-acbf473e3ead None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:27.048787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 367/717] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:29:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:27.054457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238af1-e3fa-4648-b523-0c70346f09c1 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:27.162999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238af1-e3fa-4648-b523-0c70346f09c1 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:27.201054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099308a-76d7-47f7-ab98-3d4734f0ad63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:27.201683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099308a-76d7-47f7-ab98-3d4734f0ad63 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:27.201960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099308a-76d7-47f7-ab98-3d4734f0ad63 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:27.202292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099308a-76d7-47f7-ab98-3d4734f0ad63 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:27.202946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099308a-76d7-47f7-ab98-3d4734f0ad63 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:27.203297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099308a-76d7-47f7-ab98-3d4734f0ad63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:27.204993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099308a-76d7-47f7-ab98-3d4734f0ad63 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:27.210983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 351/718] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Oct 20 14:29:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:27.217765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a82dd-a072-4178-b7a4-acbf473e3ead None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:27.325733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a82dd-a072-4178-b7a4-acbf473e3ead None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:27.361762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797da082-0f76-4201-bcc1-5fbac162a207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:27.362368 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797da082-0f76-4201-bcc1-5fbac162a207 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:27.362643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797da082-0f76-4201-bcc1-5fbac162a207 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:27.362912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797da082-0f76-4201-bcc1-5fbac162a207 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:27.363638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797da082-0f76-4201-bcc1-5fbac162a207 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:27.365021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797da082-0f76-4201-bcc1-5fbac162a207 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:27.387875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e615d-9680-47eb-a0c0-676e346a7596 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:27.388220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e615d-9680-47eb-a0c0-676e346a7596 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:27.388517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e615d-9680-47eb-a0c0-676e346a7596 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:27.388982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 368/719] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:29:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:27.389953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 352/720] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:29:27.399223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344812c-70c4-473f-a8e1-75c4c42e5fa2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:27.399223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344812c-70c4-473f-a8e1-75c4c42e5fa2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:27.399223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344812c-70c4-473f-a8e1-75c4c42e5fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:27.417562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9344812c-70c4-473f-a8e1-75c4c42e5fa2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:27.419198 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9344812c-70c4-473f-a8e1-75c4c42e5fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:27.454157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:27.511686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 369/721] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:27.621472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d537e16-af26-4146-9a31-737dd1523921 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:27.621688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d537e16-af26-4146-9a31-737dd1523921 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:27.621973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d537e16-af26-4146-9a31-737dd1523921 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:27.623429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 353/722] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:27.628724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb8f8a-e8f0-40b4-884a-3dbc45c0999d None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:27.629216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb8f8a-e8f0-40b4-884a-3dbc45c0999d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:27.629670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb8f8a-e8f0-40b4-884a-3dbc45c0999d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:27.648780 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-a2bb8f8a-e8f0-40b4-884a-3dbc45c0999d None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:27.651233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a2bb8f8a-e8f0-40b4-884a-3dbc45c0999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:27.720269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 370/723] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:27.788213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d537e16-af26-4146-9a31-737dd1523921 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:27.808783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:27.892634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d537e16-af26-4146-9a31-737dd1523921 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:27.928314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdf7a2-ed45-4a35-9552-df2deb73cd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:27.928969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdf7a2-ed45-4a35-9552-df2deb73cd5e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:27.929247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdf7a2-ed45-4a35-9552-df2deb73cd5e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:27.929516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdf7a2-ed45-4a35-9552-df2deb73cd5e None admin] PATH_INFO: `/v3/services/image` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:27.930086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdf7a2-ed45-4a35-9552-df2deb73cd5e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:27.931721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdf7a2-ed45-4a35-9552-df2deb73cd5e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-90bdf7a2-ed45-4a35-9552-df2deb73cd5e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:27.940639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:27.941597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:27.942401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 354/724] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Oct 20 14:29:27 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:27.949773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb8f8a-e8f0-40b4-884a-3dbc45c0999d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:28.000911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdf7a2-ed45-4a35-9552-df2deb73cd5e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:28.058071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:28.071814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb8f8a-e8f0-40b4-884a-3dbc45c0999d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:28.101934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:28.116131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b032f1-2a71-4e7f-87f5-9146414bdd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:28.117040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b032f1-2a71-4e7f-87f5-9146414bdd3d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:28.117713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b032f1-2a71-4e7f-87f5-9146414bdd3d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:28.118149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b032f1-2a71-4e7f-87f5-9146414bdd3d None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:28.120209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b032f1-2a71-4e7f-87f5-9146414bdd3d None admin] RBAC: Adding query filter params (name=image) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:28.121146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b032f1-2a71-4e7f-87f5-9146414bdd3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:28.123102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b032f1-2a71-4e7f-87f5-9146414bdd3d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:28.126595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdf7a2-ed45-4a35-9552-df2deb73cd5e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:28.132384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 371/725] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:28.134985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b032f1-2a71-4e7f-87f5-9146414bdd3d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:28.169270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3836e-179e-472c-a344-17b4b6a9848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:28.170247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3836e-179e-472c-a344-17b4b6a9848c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:28.170752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3836e-179e-472c-a344-17b4b6a9848c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:28.171233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3836e-179e-472c-a344-17b4b6a9848c None admin] PATH_INFO: `/v3/services/compute` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:28.172035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3836e-179e-472c-a344-17b4b6a9848c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:28.173654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3836e-179e-472c-a344-17b4b6a9848c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-80a3836e-179e-472c-a344-17b4b6a9848c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:28.183548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:28.184613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 14:29:28.185530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:28.188324 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 355/726] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:28.195431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3836e-179e-472c-a344-17b4b6a9848c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:28.258623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b032f1-2a71-4e7f-87f5-9146414bdd3d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:28.307373 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3836e-179e-472c-a344-17b4b6a9848c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:28.307723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89fcce-435c-44a6-966d-364970763370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:28.308358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89fcce-435c-44a6-966d-364970763370 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:28.308663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89fcce-435c-44a6-966d-364970763370 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:28.308939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89fcce-435c-44a6-966d-364970763370 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:28.309573 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89fcce-435c-44a6-966d-364970763370 None admin] RBAC: Adding query filter params (type=image) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:28.309877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89fcce-435c-44a6-966d-364970763370 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:28.312455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89fcce-435c-44a6-966d-364970763370 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:28.324373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 372/727] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:28.354613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296de0af-af2d-4806-9796-40fb463f3e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:28.355258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296de0af-af2d-4806-9796-40fb463f3e5b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:28.355545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296de0af-af2d-4806-9796-40fb463f3e5b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:28.355817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296de0af-af2d-4806-9796-40fb463f3e5b None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:28.356561 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296de0af-af2d-4806-9796-40fb463f3e5b None admin] RBAC: Adding query filter params (name=compute) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:28.356871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296de0af-af2d-4806-9796-40fb463f3e5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:28.358559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296de0af-af2d-4806-9796-40fb463f3e5b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:28.364998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 356/728] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:28.372380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89fcce-435c-44a6-966d-364970763370 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:28.394310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:28.499919 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89fcce-435c-44a6-966d-364970763370 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:28.542118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa58e9-4b3e-4497-aa91-4ad3f70c33d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:28.542728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa58e9-4b3e-4497-aa91-4ad3f70c33d8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:28.543010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa58e9-4b3e-4497-aa91-4ad3f70c33d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:28.543284 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa58e9-4b3e-4497-aa91-4ad3f70c33d8 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:28.543915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa58e9-4b3e-4497-aa91-4ad3f70c33d8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:28.544219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa58e9-4b3e-4497-aa91-4ad3f70c33d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:28.545672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa58e9-4b3e-4497-aa91-4ad3f70c33d8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:28.550596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 373/729] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:28.589306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee4966-6337-4caf-b48b-a2058d32a756 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:28.589625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee4966-6337-4caf-b48b-a2058d32a756 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:28.589865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee4966-6337-4caf-b48b-a2058d32a756 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:28.591559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 357/730] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:28.597424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf30040-82ff-4b5d-b65e-e911dd3e99d2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:28.597707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf30040-82ff-4b5d-b65e-e911dd3e99d2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:28.597976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf30040-82ff-4b5d-b65e-e911dd3e99d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:28.618415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-daf30040-82ff-4b5d-b65e-e911dd3e99d2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:28.620599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-daf30040-82ff-4b5d-b65e-e911dd3e99d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:28.699491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 374/731] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:28.983194 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee4966-6337-4caf-b48b-a2058d32a756 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:28.997060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:29.095807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee4966-6337-4caf-b48b-a2058d32a756 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:29.132841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8dbf5-b431-4b98-84e2-42a4a5d517db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:29.133504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8dbf5-b431-4b98-84e2-42a4a5d517db None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:29.133737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8dbf5-b431-4b98-84e2-42a4a5d517db None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:29.134014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8dbf5-b431-4b98-84e2-42a4a5d517db None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:29.135141 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8dbf5-b431-4b98-84e2-42a4a5d517db None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:29.136546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8dbf5-b431-4b98-84e2-42a4a5d517db None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-c8f8dbf5-b431-4b98-84e2-42a4a5d517db None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:29.145581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:29.146541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:29.147758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 358/732] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:29.154259 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf30040-82ff-4b5d-b65e-e911dd3e99d2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:29.166615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:29.266927 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf30040-82ff-4b5d-b65e-e911dd3e99d2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:29.303473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a4c74-2daf-42a9-a4db-388335fd9ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:29.304081 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a4c74-2daf-42a9-a4db-388335fd9ea1 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:29.304381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a4c74-2daf-42a9-a4db-388335fd9ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:29.304651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a4c74-2daf-42a9-a4db-388335fd9ea1 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:29.305282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a4c74-2daf-42a9-a4db-388335fd9ea1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:29.305583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a4c74-2daf-42a9-a4db-388335fd9ea1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:29.308030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a4c74-2daf-42a9-a4db-388335fd9ea1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:29.313342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 375/733] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:29.318891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8dbf5-b431-4b98-84e2-42a4a5d517db None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:29.404625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:29.435874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8dbf5-b431-4b98-84e2-42a4a5d517db None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:29.472093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7396cfd-0d63-401f-bba8-dc1cb44795c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:29.473098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396cfd-0d63-401f-bba8-dc1cb44795c6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:29.473596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396cfd-0d63-401f-bba8-dc1cb44795c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:29.474075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396cfd-0d63-401f-bba8-dc1cb44795c6 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:29.474911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7396cfd-0d63-401f-bba8-dc1cb44795c6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:29.475413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7396cfd-0d63-401f-bba8-dc1cb44795c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:29.477011 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7396cfd-0d63-401f-bba8-dc1cb44795c6 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:29.483957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 359/734] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:29.490071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a4c74-2daf-42a9-a4db-388335fd9ea1 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:29.578043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:29.605832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a4c74-2daf-42a9-a4db-388335fd9ea1 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:29.642112 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9d62d-2022-4be0-befb-e8115a4329d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:29.643092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9d62d-2022-4be0-befb-e8115a4329d9 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:29.643617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9d62d-2022-4be0-befb-e8115a4329d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:29.644088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9d62d-2022-4be0-befb-e8115a4329d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:29.645021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9d62d-2022-4be0-befb-e8115a4329d9 None admin] RBAC: Adding query filter params (interface=admin, service_id=016568d932fe494f83773c97dfbcb6bc, region_id=RegionOne) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:29.645530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9d62d-2022-4be0-befb-e8115a4329d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:29.647108 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9d62d-2022-4be0-befb-e8115a4329d9 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:29.653438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 376/735] 10.222.0.13 () {56 vars in 1249 bytes} [Thu Oct 20 14:29:29 2022] GET /identity/v3/endpoints?service_id=016568d932fe494f83773c97dfbcb6bc&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:29.659267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7396cfd-0d63-401f-bba8-dc1cb44795c6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:29.669889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323ed4b-c3cc-4faf-860b-001f034293dd None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:29.670168 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323ed4b-c3cc-4faf-860b-001f034293dd None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:29.670462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323ed4b-c3cc-4faf-860b-001f034293dd None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:29.672000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 377/736] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:29.677131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d33e06-74d7-4555-8563-c656bce6d1f9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:29.677410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d33e06-74d7-4555-8563-c656bce6d1f9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:29.677681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d33e06-74d7-4555-8563-c656bce6d1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:29.695421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-15d33e06-74d7-4555-8563-c656bce6d1f9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:29.697100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-15d33e06-74d7-4555-8563-c656bce6d1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:29.766677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 378/737] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:29.787009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7396cfd-0d63-401f-bba8-dc1cb44795c6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:29.824795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b49ba9-0fd5-431c-995c-4cd770e2b9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:29.825744 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b49ba9-0fd5-431c-995c-4cd770e2b9fe None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:29.826223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b49ba9-0fd5-431c-995c-4cd770e2b9fe None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:29.826692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b49ba9-0fd5-431c-995c-4cd770e2b9fe None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:29.827479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b49ba9-0fd5-431c-995c-4cd770e2b9fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:29.829146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b49ba9-0fd5-431c-995c-4cd770e2b9fe None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:29.835723 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 360/738] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Oct 20 14:29:29 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:29:30.044909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d33e06-74d7-4555-8563-c656bce6d1f9 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:30.114524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:30.149650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d33e06-74d7-4555-8563-c656bce6d1f9 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:30.184974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:30.185622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:30.185844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:30.186103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:30.186752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:30.188174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:30.190903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING keystone.server.flask.common [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 20 14:29:30.209830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.sql.core [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 20 14:29:30.209830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 14:29:30.209830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [parameters: {'id': '19204fee774748a380b7097d4da4dae7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 20 14:29:30.209830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82084) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19204fee774748a380b7097d4da4dae7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19204fee774748a380b7097d4da4dae7. Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:29:30.218734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application result.read() Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 20 14:29:30.219552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application yield resource Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise self.value Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application session.commit() Oct 20 14:29:30.220581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.session.flush() Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application compat.raise_( Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 14:29:30.221449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 14:29:30.222342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application result.read() Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:29:30.223128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [parameters: {'id': '19204fee774748a380b7097d4da4dae7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:30.223984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:30.225997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19204fee774748a380b7097d4da4dae7. Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:30.227100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 379/739] 10.222.0.13 () {60 vars in 1140 bytes} [Thu Oct 20 14:29:30 2022] POST /identity/v3/projects => generated 262 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:30.227758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b49ba9-0fd5-431c-995c-4cd770e2b9fe None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:30.230907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:30.346556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b49ba9-0fd5-431c-995c-4cd770e2b9fe None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:30.391783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99319a-96b1-41df-a6f3-402354699d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:30.392758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99319a-96b1-41df-a6f3-402354699d0d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:30.393233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99319a-96b1-41df-a6f3-402354699d0d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:30.393748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99319a-96b1-41df-a6f3-402354699d0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:30.401216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99319a-96b1-41df-a6f3-402354699d0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:30.403017 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99319a-96b1-41df-a6f3-402354699d0d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-8e99319a-96b1-41df-a6f3-402354699d0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:30.414624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:30.416161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:30.417074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:30.417074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:30.417074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:30.417074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:30.417074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:30.417074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:30.417074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:30.417074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:30.419570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 361/740] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:30.425595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:30.534404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562bf14-7c77-4313-bb78-f76716407513 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:30.542543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76582d9-de38-4918-8bc8-d1501d8e508c None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:30.542831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76582d9-de38-4918-8bc8-d1501d8e508c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:30.543115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76582d9-de38-4918-8bc8-d1501d8e508c None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:30.544743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 362/741] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:30.549837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016642e0-25bf-46a5-9443-5441f365a2c5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:30.550125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016642e0-25bf-46a5-9443-5441f365a2c5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:30.550396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016642e0-25bf-46a5-9443-5441f365a2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:30.559187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:30.572609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-016642e0-25bf-46a5-9443-5441f365a2c5 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:30.573481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220ebec-3fcc-4cb5-bdc4-2f80ac03e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:30.573789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-016642e0-25bf-46a5-9443-5441f365a2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:30.574551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220ebec-3fcc-4cb5-bdc4-2f80ac03e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:30.575027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220ebec-3fcc-4cb5-bdc4-2f80ac03e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:30.575495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220ebec-3fcc-4cb5-bdc4-2f80ac03e8d0 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:30.576390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220ebec-3fcc-4cb5-bdc4-2f80ac03e8d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:30.576919 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220ebec-3fcc-4cb5-bdc4-2f80ac03e8d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:30.578555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220ebec-3fcc-4cb5-bdc4-2f80ac03e8d0 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:30.608376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 380/742] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:30.654047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:30.664857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 363/743] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:30.915058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f511cb9-4226-435a-b1f8-a3ae63a4adf7 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:30.916013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f511cb9-4226-435a-b1f8-a3ae63a4adf7 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:30.916469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f511cb9-4226-435a-b1f8-a3ae63a4adf7 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:30.918184 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 381/744] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:30.923596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aa98a-e8ec-4b56-998a-d631dcd25b5a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:30.924000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aa98a-e8ec-4b56-998a-d631dcd25b5a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:30.924410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aa98a-e8ec-4b56-998a-d631dcd25b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:30.935578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f511cb9-4226-435a-b1f8-a3ae63a4adf7 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:30.945306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-f49aa98a-e8ec-4b56-998a-d631dcd25b5a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:30.946497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f49aa98a-e8ec-4b56-998a-d631dcd25b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:31.041463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 364/745] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:31.069908 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f511cb9-4226-435a-b1f8-a3ae63a4adf7 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:31.106023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19606-0a52-41ae-b106-f3ee45a07a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:31.106633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19606-0a52-41ae-b106-f3ee45a07a05 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:31.106912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19606-0a52-41ae-b106-f3ee45a07a05 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:31.107189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19606-0a52-41ae-b106-f3ee45a07a05 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:31.107855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19606-0a52-41ae-b106-f3ee45a07a05 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:31.109246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19606-0a52-41ae-b106-f3ee45a07a05 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:31.128527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 382/746] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:29:30 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:29:31.338854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49aa98a-e8ec-4b56-998a-d631dcd25b5a None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:31.445081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49aa98a-e8ec-4b56-998a-d631dcd25b5a None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:31.469272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:31.488416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cce6f-4fbd-43b4-834d-b88d84c915f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:31.489035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cce6f-4fbd-43b4-834d-b88d84c915f7 None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:31.489315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cce6f-4fbd-43b4-834d-b88d84c915f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:31.489586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cce6f-4fbd-43b4-834d-b88d84c915f7 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:31.490867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cce6f-4fbd-43b4-834d-b88d84c915f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:31.491615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cce6f-4fbd-43b4-834d-b88d84c915f7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:31.510911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 365/747] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:29:31 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:32.105618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627762f-8961-41a0-add4-d22eb83f485a None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:32.106148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627762f-8961-41a0-add4-d22eb83f485a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:32.106148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627762f-8961-41a0-add4-d22eb83f485a None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:32.107770 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 383/748] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:32.113189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47583c91-3deb-4088-ad3a-91ebbb546c2e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:32.113712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47583c91-3deb-4088-ad3a-91ebbb546c2e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:32.114178 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47583c91-3deb-4088-ad3a-91ebbb546c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:32.132471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-47583c91-3deb-4088-ad3a-91ebbb546c2e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:32.133929 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-47583c91-3deb-4088-ad3a-91ebbb546c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:32.211200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 366/749] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:32 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:32.477080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627762f-8961-41a0-add4-d22eb83f485a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:32.532369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:32.587400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627762f-8961-41a0-add4-d22eb83f485a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:32.621892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae9eaf-b7b0-4de4-be5f-13d7095d1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:32.622523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae9eaf-b7b0-4de4-be5f-13d7095d1001 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:32.622805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae9eaf-b7b0-4de4-be5f-13d7095d1001 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:32.623085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae9eaf-b7b0-4de4-be5f-13d7095d1001 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:32.623669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae9eaf-b7b0-4de4-be5f-13d7095d1001 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:32.625073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae9eaf-b7b0-4de4-be5f-13d7095d1001 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-77ae9eaf-b7b0-4de4-be5f-13d7095d1001 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:32.634723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:32.635689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:32.636545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 384/750] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Oct 20 14:29:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:32.642552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47583c91-3deb-4088-ad3a-91ebbb546c2e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:32.743269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47583c91-3deb-4088-ad3a-91ebbb546c2e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:32.746352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:32.783436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65314f5-0a21-409e-99c1-a74124a5be7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:32.784070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65314f5-0a21-409e-99c1-a74124a5be7b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:32.784376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65314f5-0a21-409e-99c1-a74124a5be7b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:32.784659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65314f5-0a21-409e-99c1-a74124a5be7b None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:32.785312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65314f5-0a21-409e-99c1-a74124a5be7b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:32.785622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65314f5-0a21-409e-99c1-a74124a5be7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:32.787051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65314f5-0a21-409e-99c1-a74124a5be7b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:32.797881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 367/751] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:32.799274 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae9eaf-b7b0-4de4-be5f-13d7095d1001 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:32.889391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50fbe4-0828-4bf0-9aba-7f85ef7b74e9 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:32.890043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50fbe4-0828-4bf0-9aba-7f85ef7b74e9 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:32.890519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50fbe4-0828-4bf0-9aba-7f85ef7b74e9 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:32.892308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 368/752] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:29:32.897397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f4d56-2c2a-4e6b-ae54-da8d24c13588 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:32.897922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f4d56-2c2a-4e6b-ae54-da8d24c13588 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:32.898404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f4d56-2c2a-4e6b-ae54-da8d24c13588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:32.904335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae9eaf-b7b0-4de4-be5f-13d7095d1001 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:32.916390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:32.922513 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-607f4d56-2c2a-4e6b-ae54-da8d24c13588 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:32.924031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-607f4d56-2c2a-4e6b-ae54-da8d24c13588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:32.947779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15c517-2b22-4714-8d29-3a02e573212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:32.948176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15c517-2b22-4714-8d29-3a02e573212e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:32.948485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15c517-2b22-4714-8d29-3a02e573212e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:32.948757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15c517-2b22-4714-8d29-3a02e573212e None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:32.949660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15c517-2b22-4714-8d29-3a02e573212e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:32.949969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15c517-2b22-4714-8d29-3a02e573212e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:32.951877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15c517-2b22-4714-8d29-3a02e573212e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:32.963587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 385/753] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Oct 20 14:29:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:32.965273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15c517-2b22-4714-8d29-3a02e573212e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:33.017686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 369/754] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:32 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:33.080308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15c517-2b22-4714-8d29-3a02e573212e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:33.116080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f86d86-5bf5-4bb5-8579-e92002df5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:33.116748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f86d86-5bf5-4bb5-8579-e92002df5cf6 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:33.117235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f86d86-5bf5-4bb5-8579-e92002df5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:33.117523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f86d86-5bf5-4bb5-8579-e92002df5cf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:33.118188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f86d86-5bf5-4bb5-8579-e92002df5cf6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:33.119602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f86d86-5bf5-4bb5-8579-e92002df5cf6 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:33.145234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 386/755] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:29:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:29:33.293216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f4d56-2c2a-4e6b-ae54-da8d24c13588 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:33.410531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:33.414550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f4d56-2c2a-4e6b-ae54-da8d24c13588 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:33.430469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2b94b-f8ca-4ad8-8ac7-7617378e6946 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:33.430796 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2b94b-f8ca-4ad8-8ac7-7617378e6946 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:33.431523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2b94b-f8ca-4ad8-8ac7-7617378e6946 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:33.432685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 387/756] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:29:33.437612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11664-7807-4327-b3fe-923522d8df9c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:33.437894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11664-7807-4327-b3fe-923522d8df9c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:33.438170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11664-7807-4327-b3fe-923522d8df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:33.457124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-6bd11664-7807-4327-b3fe-923522d8df9c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:33.458609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6bd11664-7807-4327-b3fe-923522d8df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:33.464376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:33.464376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:33.464376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:33.464376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:33.464719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:33.464719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:33.467125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING keystone.server.flask.common [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 20 14:29:33.482130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.sql.core [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 20 14:29:33.482130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 20 14:29:33.482130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [parameters: {'user_id': '636b0791e8444ee094c73116cc074e89', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 20 14:29:33.482130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82085) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result.read() Oct 20 14:29:33.489140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:29:33.490021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application yield resource Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise self.value Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application session.commit() Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 14:29:33.490978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.session.flush() Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application compat.raise_( Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 14:29:33.491820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:29:33.494239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result.read() Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:29:33.495084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [parameters: {'user_id': '636b0791e8444ee094c73116cc074e89', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:33.495876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 20 14:29:33.496712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 20 14:29:33.498937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 20 14:29:33.498937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 20 14:29:33.498937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 14:29:33.498937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 14:29:33.498937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 20 14:29:33.498937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:33.498937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 370/757] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Oct 20 14:29:33 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:33.498937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:33.503030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:33.563850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 388/758] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:33.624336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea04fec-709d-489d-b90a-c3a709773529 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:33.661937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c19717-0914-4054-91e9-0cf057fce368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:33.663479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c19717-0914-4054-91e9-0cf057fce368 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:33.663968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c19717-0914-4054-91e9-0cf057fce368 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:33.664509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c19717-0914-4054-91e9-0cf057fce368 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:33.673996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c19717-0914-4054-91e9-0cf057fce368 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:33.676141 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c19717-0914-4054-91e9-0cf057fce368 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-17c19717-0914-4054-91e9-0cf057fce368 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:33.690666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:33.691655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 14:29:33.692623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:33.694476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 371/759] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:29:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:33.705449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd11664-7807-4327-b3fe-923522d8df9c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:33.790248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe321c-9bef-4787-8c4e-5ce4c2e3cdc9 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:33.790731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe321c-9bef-4787-8c4e-5ce4c2e3cdc9 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:33.790731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe321c-9bef-4787-8c4e-5ce4c2e3cdc9 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:33.798980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 372/760] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:33.800527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8a34c-461d-40e6-8e32-f6dfc0ee4db8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:33.800811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8a34c-461d-40e6-8e32-f6dfc0ee4db8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:33.801256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8a34c-461d-40e6-8e32-f6dfc0ee4db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:33.818063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd11664-7807-4327-b3fe-923522d8df9c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:33.819578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-96d8a34c-461d-40e6-8e32-f6dfc0ee4db8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:33.821492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-96d8a34c-461d-40e6-8e32-f6dfc0ee4db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:33.832335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:33.856412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20534ea7-3ca3-4e09-bb98-c5acf1eaf81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:33.857026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20534ea7-3ca3-4e09-bb98-c5acf1eaf81c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:33.857304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20534ea7-3ca3-4e09-bb98-c5acf1eaf81c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:33.857573 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20534ea7-3ca3-4e09-bb98-c5acf1eaf81c None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:33.858269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20534ea7-3ca3-4e09-bb98-c5acf1eaf81c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:33.858566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20534ea7-3ca3-4e09-bb98-c5acf1eaf81c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:33.860002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20534ea7-3ca3-4e09-bb98-c5acf1eaf81c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:33.893972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 373/761] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:33.894781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8a34c-461d-40e6-8e32-f6dfc0ee4db8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:33.903013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 389/762] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Oct 20 14:29:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:34.018164 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8a34c-461d-40e6-8e32-f6dfc0ee4db8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.059512 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631e9f2-c770-479d-a5a1-322566104962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.060578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631e9f2-c770-479d-a5a1-322566104962 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.061065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631e9f2-c770-479d-a5a1-322566104962 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.061585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631e9f2-c770-479d-a5a1-322566104962 None admin] PATH_INFO: `/v3/services/image` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.062452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631e9f2-c770-479d-a5a1-322566104962 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.064110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631e9f2-c770-479d-a5a1-322566104962 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-d631e9f2-c770-479d-a5a1-322566104962 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:34.074254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:34.075240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 14:29:34.076053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:34.079398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 374/763] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Oct 20 14:29:33 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:34.085472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20534ea7-3ca3-4e09-bb98-c5acf1eaf81c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:34.108863 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:34.167264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631e9f2-c770-479d-a5a1-322566104962 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:34.183359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:34.203819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20534ea7-3ca3-4e09-bb98-c5acf1eaf81c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.241851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e746162-8074-4fdc-81c8-e413308212fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.242710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e746162-8074-4fdc-81c8-e413308212fe None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.242983 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e746162-8074-4fdc-81c8-e413308212fe None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.243255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e746162-8074-4fdc-81c8-e413308212fe None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.244669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e746162-8074-4fdc-81c8-e413308212fe None admin] RBAC: Adding query filter params (name=image) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:34.244973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e746162-8074-4fdc-81c8-e413308212fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.246346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e746162-8074-4fdc-81c8-e413308212fe None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:34.251346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 390/764] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:34.256772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e746162-8074-4fdc-81c8-e413308212fe None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:34.299930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631e9f2-c770-479d-a5a1-322566104962 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.335887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9092-0866-487d-854e-b2eb0e44f1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.336478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9092-0866-487d-854e-b2eb0e44f1a5 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.340201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9092-0866-487d-854e-b2eb0e44f1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.340201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9092-0866-487d-854e-b2eb0e44f1a5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.340201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d9092-0866-487d-854e-b2eb0e44f1a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.340201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d9092-0866-487d-854e-b2eb0e44f1a5 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-ec0d9092-0866-487d-854e-b2eb0e44f1a5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:34.348872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:34.349821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:34.350615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 375/765] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:34.356757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9092-0866-487d-854e-b2eb0e44f1a5 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:34.358086 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:34.372500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e746162-8074-4fdc-81c8-e413308212fe None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.407848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40082f-f39e-4c67-8462-f5c4ec10e573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.408479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40082f-f39e-4c67-8462-f5c4ec10e573 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.408757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40082f-f39e-4c67-8462-f5c4ec10e573 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.409028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40082f-f39e-4c67-8462-f5c4ec10e573 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.409657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40082f-f39e-4c67-8462-f5c4ec10e573 None admin] RBAC: Adding query filter params (type=image) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:34.409959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40082f-f39e-4c67-8462-f5c4ec10e573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.411322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40082f-f39e-4c67-8462-f5c4ec10e573 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:34.421813 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 391/766] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:34.422789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40082f-f39e-4c67-8462-f5c4ec10e573 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:34.444870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:34.470871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9092-0866-487d-854e-b2eb0e44f1a5 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.508174 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8605348-7e37-42ce-96e3-1682453bc457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.508807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8605348-7e37-42ce-96e3-1682453bc457 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.509745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8605348-7e37-42ce-96e3-1682453bc457 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.510142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8605348-7e37-42ce-96e3-1682453bc457 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.510774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8605348-7e37-42ce-96e3-1682453bc457 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:34.511078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8605348-7e37-42ce-96e3-1682453bc457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.512474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8605348-7e37-42ce-96e3-1682453bc457 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:34.522506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 376/767] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:34.523899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8605348-7e37-42ce-96e3-1682453bc457 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:34.531089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40082f-f39e-4c67-8462-f5c4ec10e573 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.567580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13007e39-83ee-4ae2-8700-f8a5a596b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.568181 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13007e39-83ee-4ae2-8700-f8a5a596b8bf None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.568492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13007e39-83ee-4ae2-8700-f8a5a596b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.568769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13007e39-83ee-4ae2-8700-f8a5a596b8bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.569463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13007e39-83ee-4ae2-8700-f8a5a596b8bf None admin] RBAC: Adding query filter params (interface=public, service_id=5a240df08f6042539755f04a0ba9bb74, region_id=RegionOne) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:34.569770 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13007e39-83ee-4ae2-8700-f8a5a596b8bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.571147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13007e39-83ee-4ae2-8700-f8a5a596b8bf None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:34.581495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 392/768] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Oct 20 14:29:34 2022] GET /identity/v3/endpoints?service_id=5a240df08f6042539755f04a0ba9bb74&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:34.582656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13007e39-83ee-4ae2-8700-f8a5a596b8bf None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:34.597519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:34.633467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8605348-7e37-42ce-96e3-1682453bc457 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.668078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ec470-c06f-4b7a-9758-0545c39aa524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.668718 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ec470-c06f-4b7a-9758-0545c39aa524 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.668980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ec470-c06f-4b7a-9758-0545c39aa524 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.669255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ec470-c06f-4b7a-9758-0545c39aa524 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.669887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ec470-c06f-4b7a-9758-0545c39aa524 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:34.670195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ec470-c06f-4b7a-9758-0545c39aa524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.671572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ec470-c06f-4b7a-9758-0545c39aa524 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:34.676437 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 377/769] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:34.682224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ec470-c06f-4b7a-9758-0545c39aa524 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:34.684875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:34.694343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13007e39-83ee-4ae2-8700-f8a5a596b8bf None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.732911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56848613-f29b-468a-b241-61e2cae261cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.733521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56848613-f29b-468a-b241-61e2cae261cc None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.733802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56848613-f29b-468a-b241-61e2cae261cc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.734077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56848613-f29b-468a-b241-61e2cae261cc None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.734676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56848613-f29b-468a-b241-61e2cae261cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.736518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56848613-f29b-468a-b241-61e2cae261cc None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:34.744171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 393/770] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Oct 20 14:29:34 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:29:34.791534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ec470-c06f-4b7a-9758-0545c39aa524 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.828579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f047e-b2e1-46c5-b036-cb56cb7ddc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.829203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f047e-b2e1-46c5-b036-cb56cb7ddc5a None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.829489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f047e-b2e1-46c5-b036-cb56cb7ddc5a None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.829766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f047e-b2e1-46c5-b036-cb56cb7ddc5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.830487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f047e-b2e1-46c5-b036-cb56cb7ddc5a None admin] RBAC: Adding query filter params (interface=public, service_id=3d9b59825deb47fb9bda97557879ef63, region_id=RegionOne) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:34.830799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f047e-b2e1-46c5-b036-cb56cb7ddc5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.832235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f047e-b2e1-46c5-b036-cb56cb7ddc5a None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:34.842606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 378/771] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Oct 20 14:29:34 2022] GET /identity/v3/endpoints?service_id=3d9b59825deb47fb9bda97557879ef63&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:34.844489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56848613-f29b-468a-b241-61e2cae261cc None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:34.931045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:34.954593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56848613-f29b-468a-b241-61e2cae261cc None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:34.989753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933f37e-cb29-4a6b-a780-8610dc1d9d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:34.990358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933f37e-cb29-4a6b-a780-8610dc1d9d8b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:34.990638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933f37e-cb29-4a6b-a780-8610dc1d9d8b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:34.990911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933f37e-cb29-4a6b-a780-8610dc1d9d8b None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:34.991507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933f37e-cb29-4a6b-a780-8610dc1d9d8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:34.993168 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933f37e-cb29-4a6b-a780-8610dc1d9d8b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:35.008442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 394/772] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Oct 20 14:29:34 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:29:35.400811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78330d5-d82d-4d7b-a95a-3718f0ffa8da None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:35.401200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78330d5-d82d-4d7b-a95a-3718f0ffa8da None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:35.401365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78330d5-d82d-4d7b-a95a-3718f0ffa8da None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:35.402851 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 379/773] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:35.407807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68595921-3041-4145-8f09-91d30a12b82f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:35.408235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68595921-3041-4145-8f09-91d30a12b82f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:35.408428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68595921-3041-4145-8f09-91d30a12b82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:35.426074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-68595921-3041-4145-8f09-91d30a12b82f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:35.427251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-68595921-3041-4145-8f09-91d30a12b82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:35.496180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 395/774] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:35 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:35.763532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78330d5-d82d-4d7b-a95a-3718f0ffa8da None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:35.830320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:35.867529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78330d5-d82d-4d7b-a95a-3718f0ffa8da None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:35.903710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af619610-deb8-4d81-b3dd-a03c5823d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:35.904356 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af619610-deb8-4d81-b3dd-a03c5823d18e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:35.904637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af619610-deb8-4d81-b3dd-a03c5823d18e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:35.904904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af619610-deb8-4d81-b3dd-a03c5823d18e None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:35.906040 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af619610-deb8-4d81-b3dd-a03c5823d18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:35.907464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af619610-deb8-4d81-b3dd-a03c5823d18e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:35.909183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 380/775] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:35.914642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68595921-3041-4145-8f09-91d30a12b82f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:35.983074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:36.014491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68595921-3041-4145-8f09-91d30a12b82f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:36.048169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e170d9f-501c-4211-9531-e0ece2473669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:36.048802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e170d9f-501c-4211-9531-e0ece2473669 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:36.049091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e170d9f-501c-4211-9531-e0ece2473669 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:36.049367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e170d9f-501c-4211-9531-e0ece2473669 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:36.050026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e170d9f-501c-4211-9531-e0ece2473669 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:36.051441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e170d9f-501c-4211-9531-e0ece2473669 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:36.088371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 396/776] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:29:35 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:36.912277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348038-fc5d-40d4-8580-fcb895a4de91 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:36.912847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348038-fc5d-40d4-8580-fcb895a4de91 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:36.912847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348038-fc5d-40d4-8580-fcb895a4de91 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:36.914308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 381/777] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:36.919994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cb678-ad53-4496-832c-ae6f8a8ba660 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:36.920465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cb678-ad53-4496-832c-ae6f8a8ba660 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:36.920609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cb678-ad53-4496-832c-ae6f8a8ba660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:36.939337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-462cb678-ad53-4496-832c-ae6f8a8ba660 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:36.940531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-462cb678-ad53-4496-832c-ae6f8a8ba660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:37.011826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 397/778] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:36 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:37.163412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86299c8-1fef-44a0-96f8-755451b69e0e None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:37.163581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86299c8-1fef-44a0-96f8-755451b69e0e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:37.163855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86299c8-1fef-44a0-96f8-755451b69e0e None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:37.165969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 382/779] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:37.171039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944fe7a-f47c-4676-a75a-c60f34e0db7d None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:37.171333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944fe7a-f47c-4676-a75a-c60f34e0db7d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:37.171613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944fe7a-f47c-4676-a75a-c60f34e0db7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:37.189533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0944fe7a-f47c-4676-a75a-c60f34e0db7d None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:37.190700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0944fe7a-f47c-4676-a75a-c60f34e0db7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:37.237033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:37.259664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 398/780] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:37 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:37.279380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86299c8-1fef-44a0-96f8-755451b69e0e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:37.388574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86299c8-1fef-44a0-96f8-755451b69e0e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:37.422264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98195b-5e16-46da-a285-e7475947ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:37.422813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98195b-5e16-46da-a285-e7475947ad98 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:37.423088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98195b-5e16-46da-a285-e7475947ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:37.423454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98195b-5e16-46da-a285-e7475947ad98 None admin] PATH_INFO: `/v3/services/image` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:37.424028 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98195b-5e16-46da-a285-e7475947ad98 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:37.425430 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98195b-5e16-46da-a285-e7475947ad98 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:37.427155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-dd98195b-5e16-46da-a285-e7475947ad98 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:37.436238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:37.437256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 14:29:37.438078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:37.440654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 383/781] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Oct 20 14:29:37 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:37.443158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944fe7a-f47c-4676-a75a-c60f34e0db7d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:37.529298 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98195b-5e16-46da-a285-e7475947ad98 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:37.539699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944fe7a-f47c-4676-a75a-c60f34e0db7d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:37.575082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e095c6-9edf-45f1-8304-68107af31dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:37.575705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e095c6-9edf-45f1-8304-68107af31dc2 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:37.575969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e095c6-9edf-45f1-8304-68107af31dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:37.576245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e095c6-9edf-45f1-8304-68107af31dc2 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:37.576959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e095c6-9edf-45f1-8304-68107af31dc2 None admin] RBAC: Adding query filter params (name=image) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:37.577260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e095c6-9edf-45f1-8304-68107af31dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:37.578667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e095c6-9edf-45f1-8304-68107af31dc2 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:37.583471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 399/782] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:37.589416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e095c6-9edf-45f1-8304-68107af31dc2 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:37.628067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98195b-5e16-46da-a285-e7475947ad98 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:37.637509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:37.663234 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6a493-122c-4dee-9df2-e9f48dd24be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:37.663863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6a493-122c-4dee-9df2-e9f48dd24be7 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:37.664143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6a493-122c-4dee-9df2-e9f48dd24be7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:37.664441 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6a493-122c-4dee-9df2-e9f48dd24be7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:37.665536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6a493-122c-4dee-9df2-e9f48dd24be7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:37.666917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6a493-122c-4dee-9df2-e9f48dd24be7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-4fc6a493-122c-4dee-9df2-e9f48dd24be7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:37.675869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:37.676887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:37.677814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 384/783] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:37.682879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6a493-122c-4dee-9df2-e9f48dd24be7 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:37.697120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e095c6-9edf-45f1-8304-68107af31dc2 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:37.732274 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3c8ee-307d-43a2-8527-ef4e53e8555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:37.732903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3c8ee-307d-43a2-8527-ef4e53e8555e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:37.733183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3c8ee-307d-43a2-8527-ef4e53e8555e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:37.733451 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3c8ee-307d-43a2-8527-ef4e53e8555e None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:37.734066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3c8ee-307d-43a2-8527-ef4e53e8555e None admin] RBAC: Adding query filter params (type=image) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:37.734363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3c8ee-307d-43a2-8527-ef4e53e8555e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:37.735713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3c8ee-307d-43a2-8527-ef4e53e8555e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:37.745768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 400/784] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:37.746750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3c8ee-307d-43a2-8527-ef4e53e8555e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:37.748550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:37.789564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6a493-122c-4dee-9df2-e9f48dd24be7 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:37.824304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfb4f8-8805-4685-bbd3-8970d59399c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:37.824936 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfb4f8-8805-4685-bbd3-8970d59399c6 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:37.825219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfb4f8-8805-4685-bbd3-8970d59399c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:37.825487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfb4f8-8805-4685-bbd3-8970d59399c6 None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:37.826108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfb4f8-8805-4685-bbd3-8970d59399c6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:37.826406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfb4f8-8805-4685-bbd3-8970d59399c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:37.827767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfb4f8-8805-4685-bbd3-8970d59399c6 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:37.837605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 385/785] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:37.838601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfb4f8-8805-4685-bbd3-8970d59399c6 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:37.857670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3c8ee-307d-43a2-8527-ef4e53e8555e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:37.870361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:37.896482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3658d-f0be-44d3-8b3e-cc377fabb385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:37.897085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3658d-f0be-44d3-8b3e-cc377fabb385 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:37.897368 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3658d-f0be-44d3-8b3e-cc377fabb385 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:37.897643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3658d-f0be-44d3-8b3e-cc377fabb385 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:37.898302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3658d-f0be-44d3-8b3e-cc377fabb385 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:37.900293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3658d-f0be-44d3-8b3e-cc377fabb385 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:37.922105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 401/786] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:29:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:29:37.952329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfb4f8-8805-4685-bbd3-8970d59399c6 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:37.983000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:38.001607 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3430294-0639-472c-90d8-c99ad2c8602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:38.002231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3430294-0639-472c-90d8-c99ad2c8602e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:38.002570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3430294-0639-472c-90d8-c99ad2c8602e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:38.002894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3430294-0639-472c-90d8-c99ad2c8602e None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:38.003552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3430294-0639-472c-90d8-c99ad2c8602e None admin] RBAC: Adding query filter params (type=compute) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:38.003858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3430294-0639-472c-90d8-c99ad2c8602e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:38.005275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3430294-0639-472c-90d8-c99ad2c8602e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:38.012423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 386/787] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Oct 20 14:29:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:38.016745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3658d-f0be-44d3-8b3e-cc377fabb385 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:38.125441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3658d-f0be-44d3-8b3e-cc377fabb385 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:38.159809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fa988-7ba7-4799-9f62-58d702d02bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:38.160450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa988-7ba7-4799-9f62-58d702d02bd9 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:38.160732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa988-7ba7-4799-9f62-58d702d02bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:38.161006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa988-7ba7-4799-9f62-58d702d02bd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:38.161682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fa988-7ba7-4799-9f62-58d702d02bd9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:38.163042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fa988-7ba7-4799-9f62-58d702d02bd9 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:38.186973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 402/788] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:29:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:38.307620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2968cc-82c7-4de2-97c3-40e11a1c5484 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:38.307797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2968cc-82c7-4de2-97c3-40e11a1c5484 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:38.308081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2968cc-82c7-4de2-97c3-40e11a1c5484 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:38.309668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 387/789] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:38.315353 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b45de-b910-4975-9b5b-cc734a73b3b7 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:38.315702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b45de-b910-4975-9b5b-cc734a73b3b7 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:38.315923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b45de-b910-4975-9b5b-cc734a73b3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:38.335488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d81b45de-b910-4975-9b5b-cc734a73b3b7 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:38.336737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d81b45de-b910-4975-9b5b-cc734a73b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:38.359908 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:38.428554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 403/790] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:38.703619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2968cc-82c7-4de2-97c3-40e11a1c5484 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:38.816371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2968cc-82c7-4de2-97c3-40e11a1c5484 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:38.851672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd3206-084e-4b2c-a220-8c2340a3f580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:38.852323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd3206-084e-4b2c-a220-8c2340a3f580 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:38.852659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd3206-084e-4b2c-a220-8c2340a3f580 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:38.852935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd3206-084e-4b2c-a220-8c2340a3f580 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:38.854233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd3206-084e-4b2c-a220-8c2340a3f580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:38.855765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd3206-084e-4b2c-a220-8c2340a3f580 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:38.857625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 388/791] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:38.863545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b45de-b910-4975-9b5b-cc734a73b3b7 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:38.964590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b45de-b910-4975-9b5b-cc734a73b3b7 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:38.998674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec345c-627a-43c4-951c-b983afa10f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:38.999279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec345c-627a-43c4-951c-b983afa10f16 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:38.999557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec345c-627a-43c4-951c-b983afa10f16 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:38.999831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec345c-627a-43c4-951c-b983afa10f16 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:39.000520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec345c-627a-43c4-951c-b983afa10f16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:39.001951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec345c-627a-43c4-951c-b983afa10f16 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:39.032386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 404/792] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:29:38 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:40.135696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5882-fb70-4c8f-9735-520becbee36c None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:40.136215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5882-fb70-4c8f-9735-520becbee36c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:40.136215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5882-fb70-4c8f-9735-520becbee36c None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:40.137721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 389/793] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:40.142641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2486e36-768a-4af1-8468-7efbee34d003 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:40.143051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2486e36-768a-4af1-8468-7efbee34d003 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:40.143214 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2486e36-768a-4af1-8468-7efbee34d003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:40.150978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:40.163816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e2486e36-768a-4af1-8468-7efbee34d003 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:40.165388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e2486e36-768a-4af1-8468-7efbee34d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:40.230451 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 405/794] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:40.432876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f0b25-0bec-4c22-84de-f210c37e15a5 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:40.433179 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f0b25-0bec-4c22-84de-f210c37e15a5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:40.433355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f0b25-0bec-4c22-84de-f210c37e15a5 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:40.434705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 390/795] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:40.440031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b1ecd-077d-47d8-958d-68379b407e46 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:40.440327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b1ecd-077d-47d8-958d-68379b407e46 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:40.440634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b1ecd-077d-47d8-958d-68379b407e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:40.459612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d32b1ecd-077d-47d8-958d-68379b407e46 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:40.460825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d32b1ecd-077d-47d8-958d-68379b407e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:40.495784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f0b25-0bec-4c22-84de-f210c37e15a5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:40.530735 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 406/796] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:40.556705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:40.603683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f0b25-0bec-4c22-84de-f210c37e15a5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:40.639412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:40.640013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:40.640293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:40.640597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:40.641262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:40.642636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:40.650598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.sql.core [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 14:29:40.650598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 14:29:40.650598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [parameters: {'id': 'f880e9174f6d4d4d87c892d13e83862d', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 14:29:40.650598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82085) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:29:40.657862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result.read() Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 20 14:29:40.658689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application yield resource Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application next(self.gen) Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise self.value Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application session.commit() Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 14:29:40.659644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.session.flush() Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application compat.raise_( Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 14:29:40.661048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 14:29:40.661927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application conn.query(q) Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application result.read() Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 14:29:40.662761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [parameters: {'id': 'f880e9174f6d4d4d87c892d13e83862d', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 20 14:29:40.663625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:40.665560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:40.668159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 391/797] 10.222.0.13 () {60 vars in 1137 bytes} [Thu Oct 20 14:29:40 2022] POST /identity/v3/domains => generated 190 bytes in 164 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:40.670774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b1ecd-077d-47d8-958d-68379b407e46 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:40.726196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:40.765574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b1ecd-077d-47d8-958d-68379b407e46 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:40.799467 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852dc94-3a31-4138-9dc1-d3b1cfc63693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:40.800266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852dc94-3a31-4138-9dc1-d3b1cfc63693 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:40.800626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852dc94-3a31-4138-9dc1-d3b1cfc63693 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:40.800962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852dc94-3a31-4138-9dc1-d3b1cfc63693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:40.801066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:40.811771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852dc94-3a31-4138-9dc1-d3b1cfc63693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:40.813240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852dc94-3a31-4138-9dc1-d3b1cfc63693 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-2852dc94-3a31-4138-9dc1-d3b1cfc63693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:40.828234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:40.829450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:40.830531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:40.830531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:40.830531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:40.832505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 407/798] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:40.835141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852dc94-3a31-4138-9dc1-d3b1cfc63693 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:40.912961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e7add-6a4f-4aff-92a9-c193dca5817e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:40.934037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:40.948101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852dc94-3a31-4138-9dc1-d3b1cfc63693 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:40.951989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51af32-c2b5-40e2-a62f-2b9a6d24c7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:40.952721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51af32-c2b5-40e2-a62f-2b9a6d24c7e4 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:40.953128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51af32-c2b5-40e2-a62f-2b9a6d24c7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:40.953500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51af32-c2b5-40e2-a62f-2b9a6d24c7e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:40.966413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51af32-c2b5-40e2-a62f-2b9a6d24c7e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:40.966865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-3c51af32-c2b5-40e2-a62f-2b9a6d24c7e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:40.971500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:40.972496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:40.973306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 392/799] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Oct 20 14:29:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:40.978591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51af32-c2b5-40e2-a62f-2b9a6d24c7e4 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:40.991503 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dfc58-76ef-406e-b98c-baaa46efdf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:40.992444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dfc58-76ef-406e-b98c-baaa46efdf52 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:40.992725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dfc58-76ef-406e-b98c-baaa46efdf52 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:40.993554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dfc58-76ef-406e-b98c-baaa46efdf52 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:40.994205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702dfc58-76ef-406e-b98c-baaa46efdf52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:40.994505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702dfc58-76ef-406e-b98c-baaa46efdf52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:40.995858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702dfc58-76ef-406e-b98c-baaa46efdf52 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:41.007881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 408/800] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:41.090682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51af32-c2b5-40e2-a62f-2b9a6d24c7e4 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:41.130584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df3e0b-b96f-4b4a-848f-57997911bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:41.131203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3e0b-b96f-4b4a-848f-57997911bcfb None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.131489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3e0b-b96f-4b4a-848f-57997911bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.131761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3e0b-b96f-4b4a-848f-57997911bcfb None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.132465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df3e0b-b96f-4b4a-848f-57997911bcfb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:41.132825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df3e0b-b96f-4b4a-848f-57997911bcfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:41.134387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df3e0b-b96f-4b4a-848f-57997911bcfb None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:41.144781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 393/801] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Oct 20 14:29:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:41.149857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dfc58-76ef-406e-b98c-baaa46efdf52 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:41.205779 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dc671-d19c-4bf1-b3b1-6e6293d079d0 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.206065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dc671-d19c-4bf1-b3b1-6e6293d079d0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.206345 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dc671-d19c-4bf1-b3b1-6e6293d079d0 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.207734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 394/802] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:29:41.214035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe1c6e-666f-4c11-b420-b99118ae5bd2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.214324 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe1c6e-666f-4c11-b420-b99118ae5bd2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.214598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe1c6e-666f-4c11-b420-b99118ae5bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.235230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-affe1c6e-666f-4c11-b420-b99118ae5bd2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:41.236434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-affe1c6e-666f-4c11-b420-b99118ae5bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:41.260969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dfc58-76ef-406e-b98c-baaa46efdf52 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:41.282493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:41.298800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4af9d9-0d2b-4580-b19b-644445c0b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:41.299438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4af9d9-0d2b-4580-b19b-644445c0b47f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.299714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4af9d9-0d2b-4580-b19b-644445c0b47f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.299985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4af9d9-0d2b-4580-b19b-644445c0b47f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.314826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4af9d9-0d2b-4580-b19b-644445c0b47f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:41.316287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4af9d9-0d2b-4580-b19b-644445c0b47f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:41.318498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 395/803] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-8e4af9d9-0d2b-4580-b19b-644445c0b47f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:41.331590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:41.332583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:41.333469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:41.333469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:41.333469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:41.333469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 409/804] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:41.339318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe1c6e-666f-4c11-b420-b99118ae5bd2 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:41.439870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe1c6e-666f-4c11-b420-b99118ae5bd2 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:41.472999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd97bb-7a7d-4814-9568-07bf14405c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:41.473596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd97bb-7a7d-4814-9568-07bf14405c2f None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.473877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd97bb-7a7d-4814-9568-07bf14405c2f None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.474158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd97bb-7a7d-4814-9568-07bf14405c2f None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.474795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd97bb-7a7d-4814-9568-07bf14405c2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:41.475100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd97bb-7a7d-4814-9568-07bf14405c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:41.476494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd97bb-7a7d-4814-9568-07bf14405c2f None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:41.487566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 396/805] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:41.493929 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4af9d9-0d2b-4580-b19b-644445c0b47f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:41.524964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:41.589483 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd97bb-7a7d-4814-9568-07bf14405c2f None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:41.595714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4af9d9-0d2b-4580-b19b-644445c0b47f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:41.619740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:41.628949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcfc8f-6f5d-423b-8d1c-f3c53481264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:41.629577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcfc8f-6f5d-423b-8d1c-f3c53481264e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.629855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcfc8f-6f5d-423b-8d1c-f3c53481264e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.630127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcfc8f-6f5d-423b-8d1c-f3c53481264e None admin] PATH_INFO: `/v3/users/nova` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.638500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcfc8f-6f5d-423b-8d1c-f3c53481264e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:41.639949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcfc8f-6f5d-423b-8d1c-f3c53481264e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-3afcfc8f-6f5d-423b-8d1c-f3c53481264e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:41.651862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:41.652811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:41.654294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 410/806] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Oct 20 14:29:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:41.658644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcfc8f-6f5d-423b-8d1c-f3c53481264e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:41.692578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd97bb-7a7d-4814-9568-07bf14405c2f None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:41.725625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2847-15f0-412f-b237-ecc4eb3398f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:41.726253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2847-15f0-412f-b237-ecc4eb3398f2 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.726535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2847-15f0-412f-b237-ecc4eb3398f2 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.726839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2847-15f0-412f-b237-ecc4eb3398f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.736836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa2847-15f0-412f-b237-ecc4eb3398f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-12fa2847-15f0-412f-b237-ecc4eb3398f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:41.742348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:41.743814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:41.744677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 397/807] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:29:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:41.745345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:41.750016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2847-15f0-412f-b237-ecc4eb3398f2 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:41.757429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcfc8f-6f5d-423b-8d1c-f3c53481264e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:41.791565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7aa4-d442-4a90-b21b-31522d95e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:41.792164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7aa4-d442-4a90-b21b-31522d95e95b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.792475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7aa4-d442-4a90-b21b-31522d95e95b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.792751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7aa4-d442-4a90-b21b-31522d95e95b None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.793469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7aa4-d442-4a90-b21b-31522d95e95b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:41.793775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7aa4-d442-4a90-b21b-31522d95e95b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:41.795184 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7aa4-d442-4a90-b21b-31522d95e95b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:41.812072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 411/808] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Oct 20 14:29:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:41.817730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7aa4-d442-4a90-b21b-31522d95e95b None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:41.844556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:41.854467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2847-15f0-412f-b237-ecc4eb3398f2 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:41.888612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba3196-255b-4b96-80ff-5c1ff5dbe4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:41.889235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba3196-255b-4b96-80ff-5c1ff5dbe4ab None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.889511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba3196-255b-4b96-80ff-5c1ff5dbe4ab None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.889782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba3196-255b-4b96-80ff-5c1ff5dbe4ab None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.890428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba3196-255b-4b96-80ff-5c1ff5dbe4ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:41.890725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba3196-255b-4b96-80ff-5c1ff5dbe4ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:41.892098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba3196-255b-4b96-80ff-5c1ff5dbe4ab None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:41.900934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 398/809] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:29:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:41.906565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba3196-255b-4b96-80ff-5c1ff5dbe4ab None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:41.915671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7aa4-d442-4a90-b21b-31522d95e95b None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:41.950012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5bdef-25d0-42c8-ace6-0e4bd62254ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:41.950726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5bdef-25d0-42c8-ace6-0e4bd62254ce None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:41.951009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5bdef-25d0-42c8-ace6-0e4bd62254ce None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:41.951292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5bdef-25d0-42c8-ace6-0e4bd62254ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:41.961267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5bdef-25d0-42c8-ace6-0e4bd62254ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:41.963428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5bdef-25d0-42c8-ace6-0e4bd62254ce None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:41.970011 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-a7c5bdef-25d0-42c8-ace6-0e4bd62254ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:41.978969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:41.980014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:41.980964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:41.980964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:41.980964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:41.980964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 412/810] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:41.985637 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5bdef-25d0-42c8-ace6-0e4bd62254ce None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:42.010073 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba3196-255b-4b96-80ff-5c1ff5dbe4ab None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:42.044067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eac036-6289-4e02-90e8-edcb4a5b1c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:42.044720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac036-6289-4e02-90e8-edcb4a5b1c9b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:42.044998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac036-6289-4e02-90e8-edcb4a5b1c9b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:42.045269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac036-6289-4e02-90e8-edcb4a5b1c9b None admin] PATH_INFO: `/v3/users/b73648a04aad42b8804440a70eea14b3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:42.058030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac036-6289-4e02-90e8-edcb4a5b1c9b None admin] RBAC: Authorizing `identity:get_user(user_id=b73648a04aad42b8804440a70eea14b3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:42.059463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac036-6289-4e02-90e8-edcb4a5b1c9b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:42.061084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 399/811] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:29:41 2022] GET /identity/v3/users/b73648a04aad42b8804440a70eea14b3 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:42.066645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eac036-6289-4e02-90e8-edcb4a5b1c9b None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:42.078160 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:42.083563 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5bdef-25d0-42c8-ace6-0e4bd62254ce None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:42.118051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afb2c1-346b-4b5c-8f93-61e134a6dfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:42.118652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afb2c1-346b-4b5c-8f93-61e134a6dfa3 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:42.118934 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afb2c1-346b-4b5c-8f93-61e134a6dfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:42.119209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afb2c1-346b-4b5c-8f93-61e134a6dfa3 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:42.119842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afb2c1-346b-4b5c-8f93-61e134a6dfa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:42.120149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afb2c1-346b-4b5c-8f93-61e134a6dfa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:42.121556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afb2c1-346b-4b5c-8f93-61e134a6dfa3 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:42.132955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 413/812] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:42.138869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afb2c1-346b-4b5c-8f93-61e134a6dfa3 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:42.165392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eac036-6289-4e02-90e8-edcb4a5b1c9b None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:42.195016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:42.198925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9e935-2d35-4365-b188-3b71a651beb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:42.199554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e935-2d35-4365-b188-3b71a651beb4 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:42.199837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e935-2d35-4365-b188-3b71a651beb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:42.200114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e935-2d35-4365-b188-3b71a651beb4 None admin] PATH_INFO: `/v3/projects/0dc252d60d2f4baf953a1be9839657cf` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:42.201248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9e935-2d35-4365-b188-3b71a651beb4 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc252d60d2f4baf953a1be9839657cf)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:42.202669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9e935-2d35-4365-b188-3b71a651beb4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:42.204491 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 400/813] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:29:42 2022] GET /identity/v3/projects/0dc252d60d2f4baf953a1be9839657cf => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:42.210173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9e935-2d35-4365-b188-3b71a651beb4 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:42.237103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afb2c1-346b-4b5c-8f93-61e134a6dfa3 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:42.272153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a42d9-d502-40db-b4f1-fd92d8f0a893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:42.272775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a42d9-d502-40db-b4f1-fd92d8f0a893 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:42.273051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a42d9-d502-40db-b4f1-fd92d8f0a893 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:42.273319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a42d9-d502-40db-b4f1-fd92d8f0a893 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:42.280243 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a42d9-d502-40db-b4f1-fd92d8f0a893 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:42.281722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a42d9-d502-40db-b4f1-fd92d8f0a893 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:42.291461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-f63a42d9-d502-40db-b4f1-fd92d8f0a893 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:42.292532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:42.293826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:42.294779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:42.294779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:42.294779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:42.294779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:42.294779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:42.294779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:42.294779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:42.294779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:42.294779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 414/814] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:42.299240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a42d9-d502-40db-b4f1-fd92d8f0a893 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:42.317622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9e935-2d35-4365-b188-3b71a651beb4 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:42.355606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa21937-a886-46eb-bdca-cab03bfacf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:42.356263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa21937-a886-46eb-bdca-cab03bfacf81 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:42.356572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa21937-a886-46eb-bdca-cab03bfacf81 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:42.356843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa21937-a886-46eb-bdca-cab03bfacf81 None admin] PATH_INFO: `/v3/projects/0dc252d60d2f4baf953a1be9839657cf/users/b73648a04aad42b8804440a70eea14b3/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:42.358714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa21937-a886-46eb-bdca-cab03bfacf81 None admin] RBAC: Authorizing `identity:create_grant(project_id=0dc252d60d2f4baf953a1be9839657cf, user_id=b73648a04aad42b8804440a70eea14b3, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:42.360161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa21937-a886-46eb-bdca-cab03bfacf81 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:42.373858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 401/815] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:29:42 2022] PUT /identity/v3/projects/0dc252d60d2f4baf953a1be9839657cf/users/b73648a04aad42b8804440a70eea14b3/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:29:42.400248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a42d9-d502-40db-b4f1-fd92d8f0a893 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:42.420017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:42.439284 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f73cf4-0c69-40e1-ad5e-8bb5f461178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:42.439902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f73cf4-0c69-40e1-ad5e-8bb5f461178b None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:42.440210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f73cf4-0c69-40e1-ad5e-8bb5f461178b None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:42.440588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f73cf4-0c69-40e1-ad5e-8bb5f461178b None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:42.441291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f73cf4-0c69-40e1-ad5e-8bb5f461178b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:42.441599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f73cf4-0c69-40e1-ad5e-8bb5f461178b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:42.443080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f73cf4-0c69-40e1-ad5e-8bb5f461178b None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:42.456116 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 415/816] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:42.463702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa21937-a886-46eb-bdca-cab03bfacf81 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:42.569798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa21937-a886-46eb-bdca-cab03bfacf81 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:42.602954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae190-0ef6-4d09-b8f3-eb1551e113a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:42.603607 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae190-0ef6-4d09-b8f3-eb1551e113a4 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:42.603885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae190-0ef6-4d09-b8f3-eb1551e113a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:42.604158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae190-0ef6-4d09-b8f3-eb1551e113a4 None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/9647f8caa6cd4208800e2371fdf28301/roles/1430408cb7654b4889c5df352e3c9b26` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:42.606289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ae190-0ef6-4d09-b8f3-eb1551e113a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=9647f8caa6cd4208800e2371fdf28301, role_id=1430408cb7654b4889c5df352e3c9b26)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:42.607741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ae190-0ef6-4d09-b8f3-eb1551e113a4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:42.615885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:42.622439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 402/817] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:29:42 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/9647f8caa6cd4208800e2371fdf28301/roles/1430408cb7654b4889c5df352e3c9b26 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:29:44.518509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecf7c0-4f56-4d86-b2c5-53fbca2f9f94 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:44.519083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecf7c0-4f56-4d86-b2c5-53fbca2f9f94 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:44.519083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecf7c0-4f56-4d86-b2c5-53fbca2f9f94 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:44.520405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 416/818] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:44.526612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e4d99-0ef4-4123-8a81-9d22e1d15ba9 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:44.527046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e4d99-0ef4-4123-8a81-9d22e1d15ba9 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:44.527148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e4d99-0ef4-4123-8a81-9d22e1d15ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:44.545189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-697e4d99-0ef4-4123-8a81-9d22e1d15ba9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:44.546368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-697e4d99-0ef4-4123-8a81-9d22e1d15ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:44.612139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 403/819] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:44.750401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76c3d3-6bd8-403e-ba92-4b45de1fb5bd None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:44.750681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76c3d3-6bd8-403e-ba92-4b45de1fb5bd None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:44.750951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76c3d3-6bd8-403e-ba92-4b45de1fb5bd None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:44.752312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 417/820] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:44.757359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecabfe-5253-41d5-9730-e13e667df63c None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:44.757645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecabfe-5253-41d5-9730-e13e667df63c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:44.757920 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecabfe-5253-41d5-9730-e13e667df63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:44.777065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-cdecabfe-5253-41d5-9730-e13e667df63c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:44.778241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cdecabfe-5253-41d5-9730-e13e667df63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:44.842741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 404/821] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:44.882653 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76c3d3-6bd8-403e-ba92-4b45de1fb5bd None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:44.983719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76c3d3-6bd8-403e-ba92-4b45de1fb5bd None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.017062 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fdcb2-837c-4242-8cb2-a4955c07a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.017681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fdcb2-837c-4242-8cb2-a4955c07a50f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.017960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fdcb2-837c-4242-8cb2-a4955c07a50f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.018235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fdcb2-837c-4242-8cb2-a4955c07a50f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.027933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fdcb2-837c-4242-8cb2-a4955c07a50f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-f20fdcb2-837c-4242-8cb2-a4955c07a50f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:45.032758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:45.033880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:45.034761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 418/822] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:29:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:45.039939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecabfe-5253-41d5-9730-e13e667df63c None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.096048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:45.118613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fdcb2-837c-4242-8cb2-a4955c07a50f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.138674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecabfe-5253-41d5-9730-e13e667df63c None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.158089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:45.172534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b897db-0948-4cb3-92bc-f1bcfa7e5ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.173139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b897db-0948-4cb3-92bc-f1bcfa7e5ddd None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.173417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b897db-0948-4cb3-92bc-f1bcfa7e5ddd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.173689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b897db-0948-4cb3-92bc-f1bcfa7e5ddd None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.174345 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b897db-0948-4cb3-92bc-f1bcfa7e5ddd None admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:45.174647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b897db-0948-4cb3-92bc-f1bcfa7e5ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:45.176015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b897db-0948-4cb3-92bc-f1bcfa7e5ddd None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:45.184494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 405/823] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:45.190298 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b897db-0948-4cb3-92bc-f1bcfa7e5ddd None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.223392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fdcb2-837c-4242-8cb2-a4955c07a50f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.256818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82326a42-9a1a-4633-8102-46555743c769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.257440 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82326a42-9a1a-4633-8102-46555743c769 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.257714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82326a42-9a1a-4633-8102-46555743c769 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.257983 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82326a42-9a1a-4633-8102-46555743c769 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.268141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82326a42-9a1a-4633-8102-46555743c769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-82326a42-9a1a-4633-8102-46555743c769 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:29:45.273066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:45.274087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:45.275443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 419/824] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:45.280661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82326a42-9a1a-4633-8102-46555743c769 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.290871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b897db-0948-4cb3-92bc-f1bcfa7e5ddd None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.309543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:45.323967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6be571-1577-4033-a0f1-21c8f828fd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.324620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6be571-1577-4033-a0f1-21c8f828fd30 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.324899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6be571-1577-4033-a0f1-21c8f828fd30 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.325172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6be571-1577-4033-a0f1-21c8f828fd30 None admin] PATH_INFO: `/v3/users/b73648a04aad42b8804440a70eea14b3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.326276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6be571-1577-4033-a0f1-21c8f828fd30 None admin] RBAC: Authorizing `identity:get_user(user_id=b73648a04aad42b8804440a70eea14b3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:45.327700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6be571-1577-4033-a0f1-21c8f828fd30 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:45.329278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 406/825] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/users/b73648a04aad42b8804440a70eea14b3 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:45.335101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6be571-1577-4033-a0f1-21c8f828fd30 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.376261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:45.380032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82326a42-9a1a-4633-8102-46555743c769 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.414317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b236a-edfe-4bab-8bc6-3c761b20fe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.415354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b236a-edfe-4bab-8bc6-3c761b20fe48 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.415625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b236a-edfe-4bab-8bc6-3c761b20fe48 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.415972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b236a-edfe-4bab-8bc6-3c761b20fe48 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.416649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b236a-edfe-4bab-8bc6-3c761b20fe48 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:45.416948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b236a-edfe-4bab-8bc6-3c761b20fe48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:45.418309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b236a-edfe-4bab-8bc6-3c761b20fe48 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:45.425986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 420/826] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:45.432584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b236a-edfe-4bab-8bc6-3c761b20fe48 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.435011 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6be571-1577-4033-a0f1-21c8f828fd30 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.470402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c293bf-f53f-41d5-a2ad-bea02240bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.471037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c293bf-f53f-41d5-a2ad-bea02240bf3c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.471333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c293bf-f53f-41d5-a2ad-bea02240bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.471593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c293bf-f53f-41d5-a2ad-bea02240bf3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.532596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:45.545880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c293bf-f53f-41d5-a2ad-bea02240bf3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:45.547374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c293bf-f53f-41d5-a2ad-bea02240bf3c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:45.643578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b236a-edfe-4bab-8bc6-3c761b20fe48 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.647095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 407/827] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 20 14:29:45.653112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c293bf-f53f-41d5-a2ad-bea02240bf3c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.677165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2acdb-3d06-46d7-9bcc-89fc8ffe3a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.677730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2acdb-3d06-46d7-9bcc-89fc8ffe3a5c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.678008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2acdb-3d06-46d7-9bcc-89fc8ffe3a5c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.678281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2acdb-3d06-46d7-9bcc-89fc8ffe3a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.688704 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2acdb-3d06-46d7-9bcc-89fc8ffe3a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:45.690161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2acdb-3d06-46d7-9bcc-89fc8ffe3a5c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-afa2acdb-3d06-46d7-9bcc-89fc8ffe3a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:45.704305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:45.705382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:45.706308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:45.706308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:45.706308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:45.706308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 421/828] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:45.711063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2acdb-3d06-46d7-9bcc-89fc8ffe3a5c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.725346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:45.739256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:45.752244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c293bf-f53f-41d5-a2ad-bea02240bf3c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.785355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea7195-24be-4a06-a31a-437f2a9af632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.785980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea7195-24be-4a06-a31a-437f2a9af632 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.786267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea7195-24be-4a06-a31a-437f2a9af632 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.786540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea7195-24be-4a06-a31a-437f2a9af632 None admin] PATH_INFO: `/v3/projects/0dc252d60d2f4baf953a1be9839657cf` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.787648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea7195-24be-4a06-a31a-437f2a9af632 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc252d60d2f4baf953a1be9839657cf)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:45.789132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea7195-24be-4a06-a31a-437f2a9af632 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:45.790818 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 408/829] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/projects/0dc252d60d2f4baf953a1be9839657cf => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:45.796856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea7195-24be-4a06-a31a-437f2a9af632 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.809293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2acdb-3d06-46d7-9bcc-89fc8ffe3a5c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.843707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e8f51-0e66-48d4-897c-da81a82e0d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.844309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e8f51-0e66-48d4-897c-da81a82e0d61 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.844619 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e8f51-0e66-48d4-897c-da81a82e0d61 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.844893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e8f51-0e66-48d4-897c-da81a82e0d61 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.845541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e8f51-0e66-48d4-897c-da81a82e0d61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:45.845847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e8f51-0e66-48d4-897c-da81a82e0d61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:45.847205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e8f51-0e66-48d4-897c-da81a82e0d61 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:45.858618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 422/830] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:45.864205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e8f51-0e66-48d4-897c-da81a82e0d61 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:45.895341 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea7195-24be-4a06-a31a-437f2a9af632 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.928198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbcda4-8b79-401e-bd2f-567db195eadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.928761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbcda4-8b79-401e-bd2f-567db195eadc None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.929036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbcda4-8b79-401e-bd2f-567db195eadc None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.929305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbcda4-8b79-401e-bd2f-567db195eadc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:45.930014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbcda4-8b79-401e-bd2f-567db195eadc None admin] RBAC: Adding query filter params (role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.project.id=0dc252d60d2f4baf953a1be9839657cf, user.id=b73648a04aad42b8804440a70eea14b3) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:45.930363 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbcda4-8b79-401e-bd2f-567db195eadc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:45.931235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:45.931800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbcda4-8b79-401e-bd2f-567db195eadc None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:45.937438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 409/831] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/role_assignments?user.id=b73648a04aad42b8804440a70eea14b3&scope.project.id=0dc252d60d2f4baf953a1be9839657cf&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:45.961311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e8f51-0e66-48d4-897c-da81a82e0d61 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:45.997456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207a7b9-792c-4063-b77b-4c280188b57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:45.998103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207a7b9-792c-4063-b77b-4c280188b57c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:45.998378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207a7b9-792c-4063-b77b-4c280188b57c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:45.998643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207a7b9-792c-4063-b77b-4c280188b57c None admin] PATH_INFO: `/v3/users/nova` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:46.006913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207a7b9-792c-4063-b77b-4c280188b57c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:46.008472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207a7b9-792c-4063-b77b-4c280188b57c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-6207a7b9-792c-4063-b77b-4c280188b57c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:29:46.020831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:46.021962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:29:46.022900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 423/832] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Oct 20 14:29:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:29:46.028193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbcda4-8b79-401e-bd2f-567db195eadc None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:46.051639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:46.134345 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbcda4-8b79-401e-bd2f-567db195eadc None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:46.168376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca2543-d146-4d46-91b8-31a9f4253a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:46.168980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca2543-d146-4d46-91b8-31a9f4253a4e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:46.169257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca2543-d146-4d46-91b8-31a9f4253a4e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:46.169528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca2543-d146-4d46-91b8-31a9f4253a4e None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:46.170240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca2543-d146-4d46-91b8-31a9f4253a4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:46.170543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca2543-d146-4d46-91b8-31a9f4253a4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:46.171961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca2543-d146-4d46-91b8-31a9f4253a4e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:46.189738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 410/833] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Oct 20 14:29:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:46.195584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207a7b9-792c-4063-b77b-4c280188b57c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:46.293449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207a7b9-792c-4063-b77b-4c280188b57c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:46.322437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:46.327885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71573f-5630-4fa0-89b1-e8175b978be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:46.328529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71573f-5630-4fa0-89b1-e8175b978be4 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:46.328815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71573f-5630-4fa0-89b1-e8175b978be4 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:46.329091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71573f-5630-4fa0-89b1-e8175b978be4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:46.400466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71573f-5630-4fa0-89b1-e8175b978be4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:46.401898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71573f-5630-4fa0-89b1-e8175b978be4 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:46.495841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 424/834] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:29:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 14:29:46.501459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca2543-d146-4d46-91b8-31a9f4253a4e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:46.576870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:46.598894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca2543-d146-4d46-91b8-31a9f4253a4e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:46.632306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc3c32-40b9-4fd4-aab8-d9c557aedc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:46.632960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3c32-40b9-4fd4-aab8-d9c557aedc58 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:46.633236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3c32-40b9-4fd4-aab8-d9c557aedc58 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:46.634166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3c32-40b9-4fd4-aab8-d9c557aedc58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:46.643813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc3c32-40b9-4fd4-aab8-d9c557aedc58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:46.645269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc3c32-40b9-4fd4-aab8-d9c557aedc58 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-fccc3c32-40b9-4fd4-aab8-d9c557aedc58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:29:46.660686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:46.661730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:29:46.662597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:29:46.662597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:29:46.662597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:46.662597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 411/835] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:46.667479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71573f-5630-4fa0-89b1-e8175b978be4 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:46.697223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:46.763936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71573f-5630-4fa0-89b1-e8175b978be4 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:46.796387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c560c3-0fda-4da1-9dfa-c1f5acbdeca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:46.796988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c560c3-0fda-4da1-9dfa-c1f5acbdeca5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:46.797265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c560c3-0fda-4da1-9dfa-c1f5acbdeca5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:46.797539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c560c3-0fda-4da1-9dfa-c1f5acbdeca5 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:46.798169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c560c3-0fda-4da1-9dfa-c1f5acbdeca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:46.798471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c560c3-0fda-4da1-9dfa-c1f5acbdeca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:46.799824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c560c3-0fda-4da1-9dfa-c1f5acbdeca5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:46.810740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 425/836] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:46.816390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc3c32-40b9-4fd4-aab8-d9c557aedc58 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:46.915759 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc3c32-40b9-4fd4-aab8-d9c557aedc58 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:46.942327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:46.949141 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbb1c8-05fe-4744-8391-03ec18a49719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:46.949764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbb1c8-05fe-4744-8391-03ec18a49719 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:46.950045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbb1c8-05fe-4744-8391-03ec18a49719 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:46.950319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbb1c8-05fe-4744-8391-03ec18a49719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:46.956782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbb1c8-05fe-4744-8391-03ec18a49719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:46.958220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbb1c8-05fe-4744-8391-03ec18a49719 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-33fbb1c8-05fe-4744-8391-03ec18a49719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:29:46.969351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:29:46.970389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:29:46.971251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:29:46.971251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:29:46.971251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:29:46.971251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:29:46.971251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:29:46.971251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:29:46.971251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:29:46.971251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:29:46.971251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 412/837] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:29:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:29:46.976301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c560c3-0fda-4da1-9dfa-c1f5acbdeca5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:47.064040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:47.074143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c560c3-0fda-4da1-9dfa-c1f5acbdeca5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:47.105927 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d9caad-a799-48b3-badc-ba3d69520962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:47.106523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9caad-a799-48b3-badc-ba3d69520962 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:47.106802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9caad-a799-48b3-badc-ba3d69520962 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:47.107075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9caad-a799-48b3-badc-ba3d69520962 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:47.107759 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d9caad-a799-48b3-badc-ba3d69520962 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:47.108064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d9caad-a799-48b3-badc-ba3d69520962 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:47.109518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d9caad-a799-48b3-badc-ba3d69520962 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:47.121911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 426/838] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:29:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:29:47.127849 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbb1c8-05fe-4744-8391-03ec18a49719 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:47.228170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbb1c8-05fe-4744-8391-03ec18a49719 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:47.261725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc29650-923e-4d2d-8f0a-6e3574a6d28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:47.262316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc29650-923e-4d2d-8f0a-6e3574a6d28b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:47.262590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc29650-923e-4d2d-8f0a-6e3574a6d28b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:47.262859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc29650-923e-4d2d-8f0a-6e3574a6d28b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:47.263563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc29650-923e-4d2d-8f0a-6e3574a6d28b None admin] RBAC: Adding query filter params (role.id=1430408cb7654b4889c5df352e3c9b26, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=9647f8caa6cd4208800e2371fdf28301) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:47.263864 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc29650-923e-4d2d-8f0a-6e3574a6d28b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:47.265315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc29650-923e-4d2d-8f0a-6e3574a6d28b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:47.270866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 413/839] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:29:47 2022] GET /identity/v3/role_assignments?user.id=9647f8caa6cd4208800e2371fdf28301&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=1430408cb7654b4889c5df352e3c9b26 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:48.124954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729680d-3597-451e-badb-f1f96d46e43a None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:48.125507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729680d-3597-451e-badb-f1f96d46e43a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:48.125507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729680d-3597-451e-badb-f1f96d46e43a None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:48.126799 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 427/840] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:48.131900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb2640-6792-41db-80cc-c7e3d015496a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:48.132314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb2640-6792-41db-80cc-c7e3d015496a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:48.132484 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb2640-6792-41db-80cc-c7e3d015496a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:48.150419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-cafb2640-6792-41db-80cc-c7e3d015496a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:48.151578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cafb2640-6792-41db-80cc-c7e3d015496a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:48.200516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:48.217545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 414/841] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:48.492020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729680d-3597-451e-badb-f1f96d46e43a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:48.588357 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729680d-3597-451e-badb-f1f96d46e43a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:48.620364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9109eb-aed5-42d1-9324-946c09034454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:48.620980 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9109eb-aed5-42d1-9324-946c09034454 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:48.621256 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9109eb-aed5-42d1-9324-946c09034454 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:48.621522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9109eb-aed5-42d1-9324-946c09034454 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:48.622588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9109eb-aed5-42d1-9324-946c09034454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:48.623995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9109eb-aed5-42d1-9324-946c09034454 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:48.625523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 428/842] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:48.631111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb2640-6792-41db-80cc-c7e3d015496a None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:48.725761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb2640-6792-41db-80cc-c7e3d015496a None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:48.759122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6299cd0-d120-4e5d-8143-37b36edc01d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:48.759852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6299cd0-d120-4e5d-8143-37b36edc01d3 None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:48.760136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6299cd0-d120-4e5d-8143-37b36edc01d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:48.760439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6299cd0-d120-4e5d-8143-37b36edc01d3 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:48.761085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6299cd0-d120-4e5d-8143-37b36edc01d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:48.762511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6299cd0-d120-4e5d-8143-37b36edc01d3 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:48.783350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 415/843] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:29:48 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:29:50.922069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99601-27b8-42c4-97e9-f0486dada2d2 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:50.922621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99601-27b8-42c4-97e9-f0486dada2d2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:50.922621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99601-27b8-42c4-97e9-f0486dada2d2 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:50.923925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 429/844] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:50.929015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058132bd-c51f-4543-b825-d7b31252f903 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:50.929438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058132bd-c51f-4543-b825-d7b31252f903 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:50.929576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058132bd-c51f-4543-b825-d7b31252f903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:50.949159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-058132bd-c51f-4543-b825-d7b31252f903 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:50.950327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-058132bd-c51f-4543-b825-d7b31252f903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:50.985792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:51.019217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 416/845] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:51.283989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a99601-27b8-42c4-97e9-f0486dada2d2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:51.305826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:51.381517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a99601-27b8-42c4-97e9-f0486dada2d2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:51.413221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bff648-edcb-47be-95aa-af597894b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:51.413839 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bff648-edcb-47be-95aa-af597894b02a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:51.414118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bff648-edcb-47be-95aa-af597894b02a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:51.414878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bff648-edcb-47be-95aa-af597894b02a None admin] PATH_INFO: `/v3/roles/dff349185bc94cb4a0e0f0c40425862f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:51.416271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bff648-edcb-47be-95aa-af597894b02a None admin] RBAC: Authorizing `identity:get_role(role_id=dff349185bc94cb4a0e0f0c40425862f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:51.417658 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bff648-edcb-47be-95aa-af597894b02a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:51.418957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 430/846] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:29:51 2022] GET /identity/v3/roles/dff349185bc94cb4a0e0f0c40425862f => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:51.424775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058132bd-c51f-4543-b825-d7b31252f903 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:51.520482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058132bd-c51f-4543-b825-d7b31252f903 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:51.667653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa77be-c2a6-48c1-b070-7d9fcd59230e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:51.668175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa77be-c2a6-48c1-b070-7d9fcd59230e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:51.668478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa77be-c2a6-48c1-b070-7d9fcd59230e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:51.668748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa77be-c2a6-48c1-b070-7d9fcd59230e None admin] PATH_INFO: `/v3/users/3216989253754c3081dd38409483b519` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:51.681521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa77be-c2a6-48c1-b070-7d9fcd59230e None admin] RBAC: Authorizing `identity:get_user(user_id=3216989253754c3081dd38409483b519)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:51.682932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa77be-c2a6-48c1-b070-7d9fcd59230e None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:51.684494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 417/847] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:29:51 2022] GET /identity/v3/users/3216989253754c3081dd38409483b519 => generated 286 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:51.690257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bff648-edcb-47be-95aa-af597894b02a None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:51.773708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:51.785971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bff648-edcb-47be-95aa-af597894b02a None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:51.818042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca08e42-c47b-4db0-b3df-001208a4daea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:51.818650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca08e42-c47b-4db0-b3df-001208a4daea None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:51.818931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca08e42-c47b-4db0-b3df-001208a4daea None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:51.819203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca08e42-c47b-4db0-b3df-001208a4daea None admin] PATH_INFO: `/v3/projects/0dc252d60d2f4baf953a1be9839657cf` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:51.820285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca08e42-c47b-4db0-b3df-001208a4daea None admin] RBAC: Authorizing `identity:get_project(project_id=0dc252d60d2f4baf953a1be9839657cf)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:51.821756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca08e42-c47b-4db0-b3df-001208a4daea None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:51.823340 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 431/848] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:29:51 2022] GET /identity/v3/projects/0dc252d60d2f4baf953a1be9839657cf => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:51.829121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa77be-c2a6-48c1-b070-7d9fcd59230e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:51.857620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:51.926437 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa77be-c2a6-48c1-b070-7d9fcd59230e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:51.958476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf347b-fc83-4834-8831-ec5c793b95de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:51.959116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf347b-fc83-4834-8831-ec5c793b95de None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:51.959389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf347b-fc83-4834-8831-ec5c793b95de None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:51.959717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf347b-fc83-4834-8831-ec5c793b95de None admin] PATH_INFO: `/v3/projects/0dc252d60d2f4baf953a1be9839657cf/users/3216989253754c3081dd38409483b519/roles/dff349185bc94cb4a0e0f0c40425862f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:51.962402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf347b-fc83-4834-8831-ec5c793b95de None admin] RBAC: Authorizing `identity:create_grant(project_id=0dc252d60d2f4baf953a1be9839657cf, user_id=3216989253754c3081dd38409483b519, role_id=dff349185bc94cb4a0e0f0c40425862f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:51.963825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf347b-fc83-4834-8831-ec5c793b95de None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:51.976236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 418/849] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:29:51 2022] PUT /identity/v3/projects/0dc252d60d2f4baf953a1be9839657cf/users/3216989253754c3081dd38409483b519/roles/dff349185bc94cb4a0e0f0c40425862f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:29:54.082672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e622c8-937a-4e0b-b232-e949ce6fb983 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:54.083205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e622c8-937a-4e0b-b232-e949ce6fb983 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:54.083205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e622c8-937a-4e0b-b232-e949ce6fb983 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:54.084567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 432/850] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:54.090198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96373b9-e0c4-441b-905b-1072cb3f64c1 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:54.090619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96373b9-e0c4-441b-905b-1072cb3f64c1 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:54.090725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96373b9-e0c4-441b-905b-1072cb3f64c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:54.108652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d96373b9-e0c4-441b-905b-1072cb3f64c1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:54.109802 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d96373b9-e0c4-441b-905b-1072cb3f64c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:54.177272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 419/851] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:29:54.446377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e622c8-937a-4e0b-b232-e949ce6fb983 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:54.542596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e622c8-937a-4e0b-b232-e949ce6fb983 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:54.574395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e7055-2f6c-464b-9cc9-a5d4fd98b393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:54.575003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e7055-2f6c-464b-9cc9-a5d4fd98b393 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:54.575281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e7055-2f6c-464b-9cc9-a5d4fd98b393 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:54.575554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e7055-2f6c-464b-9cc9-a5d4fd98b393 None admin] PATH_INFO: `/v3/roles/dff349185bc94cb4a0e0f0c40425862f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:54.576938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e7055-2f6c-464b-9cc9-a5d4fd98b393 None admin] RBAC: Authorizing `identity:get_role(role_id=dff349185bc94cb4a0e0f0c40425862f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:54.578288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e7055-2f6c-464b-9cc9-a5d4fd98b393 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:54.579560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 433/852] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:29:54 2022] GET /identity/v3/roles/dff349185bc94cb4a0e0f0c40425862f => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:54.585269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96373b9-e0c4-441b-905b-1072cb3f64c1 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:54.605173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:54.684283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96373b9-e0c4-441b-905b-1072cb3f64c1 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:54.716498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d389ca-5140-4248-bd76-8ab4ccd02cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:54.717108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d389ca-5140-4248-bd76-8ab4ccd02cdb None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:54.717389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d389ca-5140-4248-bd76-8ab4ccd02cdb None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:54.717660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d389ca-5140-4248-bd76-8ab4ccd02cdb None admin] PATH_INFO: `/v3/users/3216989253754c3081dd38409483b519` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:54.718747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d389ca-5140-4248-bd76-8ab4ccd02cdb None admin] RBAC: Authorizing `identity:get_user(user_id=3216989253754c3081dd38409483b519)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:54.720162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d389ca-5140-4248-bd76-8ab4ccd02cdb None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:54.721736 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 420/853] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:29:54 2022] GET /identity/v3/users/3216989253754c3081dd38409483b519 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:54.727445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e7055-2f6c-464b-9cc9-a5d4fd98b393 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:54.751810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:54.822339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e7055-2f6c-464b-9cc9-a5d4fd98b393 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:54.854319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a470e-a7fe-4abd-b40c-4f3a9d42124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:54.854900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a470e-a7fe-4abd-b40c-4f3a9d42124f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:54.855228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a470e-a7fe-4abd-b40c-4f3a9d42124f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:54.855450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a470e-a7fe-4abd-b40c-4f3a9d42124f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:54.918467 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a470e-a7fe-4abd-b40c-4f3a9d42124f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:54.919880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a470e-a7fe-4abd-b40c-4f3a9d42124f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:54.941775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:55.014558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 434/854] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:29:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 14:29:55.020388 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d389ca-5140-4248-bd76-8ab4ccd02cdb None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:55.102104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:55.117167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d389ca-5140-4248-bd76-8ab4ccd02cdb None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:55.148534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a701a-cf90-4969-ad48-a1b7746bc144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:55.149726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a701a-cf90-4969-ad48-a1b7746bc144 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:55.149997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a701a-cf90-4969-ad48-a1b7746bc144 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:55.150266 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a701a-cf90-4969-ad48-a1b7746bc144 None admin] PATH_INFO: `/v3/projects/0dc252d60d2f4baf953a1be9839657cf` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:55.151354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a701a-cf90-4969-ad48-a1b7746bc144 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc252d60d2f4baf953a1be9839657cf)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:55.152803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a701a-cf90-4969-ad48-a1b7746bc144 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:55.154364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 421/855] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:29:55 2022] GET /identity/v3/projects/0dc252d60d2f4baf953a1be9839657cf => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:55.160561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a470e-a7fe-4abd-b40c-4f3a9d42124f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:55.254956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a470e-a7fe-4abd-b40c-4f3a9d42124f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:55.277612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:55.288117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36903f-e116-44c9-84d7-19837a4199c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:55.288741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36903f-e116-44c9-84d7-19837a4199c8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:55.289025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36903f-e116-44c9-84d7-19837a4199c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:55.289479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36903f-e116-44c9-84d7-19837a4199c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:55.290311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36903f-e116-44c9-84d7-19837a4199c8 None admin] RBAC: Adding query filter params (role.id=dff349185bc94cb4a0e0f0c40425862f, scope.project.id=0dc252d60d2f4baf953a1be9839657cf, user.id=3216989253754c3081dd38409483b519) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:29:55.290619 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36903f-e116-44c9-84d7-19837a4199c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:55.292030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36903f-e116-44c9-84d7-19837a4199c8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:55.297536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 435/856] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:29:55 2022] GET /identity/v3/role_assignments?user.id=3216989253754c3081dd38409483b519&scope.project.id=0dc252d60d2f4baf953a1be9839657cf&role.id=dff349185bc94cb4a0e0f0c40425862f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:57.455681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ddcd5-d3af-4f1a-aafb-1c7ce6deb8e5 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:57.456223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ddcd5-d3af-4f1a-aafb-1c7ce6deb8e5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:57.456223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ddcd5-d3af-4f1a-aafb-1c7ce6deb8e5 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:57.457580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 422/857] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:29:57.463648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4827-6a3f-4de6-9444-1d0d91e0b53c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:57.464070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4827-6a3f-4de6-9444-1d0d91e0b53c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:57.464186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4827-6a3f-4de6-9444-1d0d91e0b53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:57.481637 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4f8c4827-6a3f-4de6-9444-1d0d91e0b53c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:29:57.482795 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4f8c4827-6a3f-4de6-9444-1d0d91e0b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:29:57.547262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 436/858] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:29:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:29:57.813817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ddcd5-d3af-4f1a-aafb-1c7ce6deb8e5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:57.912082 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ddcd5-d3af-4f1a-aafb-1c7ce6deb8e5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:57.945440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcb47c-0b23-4824-ad1e-88accbba99ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:57.946060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb47c-0b23-4824-ad1e-88accbba99ca None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:57.946340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb47c-0b23-4824-ad1e-88accbba99ca None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:57.946610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb47c-0b23-4824-ad1e-88accbba99ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:57.947738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcb47c-0b23-4824-ad1e-88accbba99ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:57.949195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcb47c-0b23-4824-ad1e-88accbba99ca None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:57.950737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 423/859] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:29:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:29:57.956499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4827-6a3f-4de6-9444-1d0d91e0b53c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:29:58.050197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4827-6a3f-4de6-9444-1d0d91e0b53c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:29:58.066602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:29:58.083463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e5801-a613-4ad6-b76e-b6509212beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:29:58.084060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5801-a613-4ad6-b76e-b6509212beb1 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:29:58.084337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5801-a613-4ad6-b76e-b6509212beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:29:58.084638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5801-a613-4ad6-b76e-b6509212beb1 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:29:58.085291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e5801-a613-4ad6-b76e-b6509212beb1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:29:58.087324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e5801-a613-4ad6-b76e-b6509212beb1 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:29:58.112196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 437/860] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:29:57 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:30:00.243750 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57257db-f3f4-4d3b-a2e3-85e47dc42996 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:00.250378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57257db-f3f4-4d3b-a2e3-85e47dc42996 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:00.250378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57257db-f3f4-4d3b-a2e3-85e47dc42996 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:00.250378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 424/861] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:30:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:30:00.253313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0663838-5980-4d44-b51d-5fcf13e665ea None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:00.253752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0663838-5980-4d44-b51d-5fcf13e665ea None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:00.253849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0663838-5980-4d44-b51d-5fcf13e665ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:00.272035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c0663838-5980-4d44-b51d-5fcf13e665ea None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:30:00.273241 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c0663838-5980-4d44-b51d-5fcf13e665ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:30:00.335391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 438/862] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:30:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:30:00.606514 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57257db-f3f4-4d3b-a2e3-85e47dc42996 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:00.630698 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:00.704577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57257db-f3f4-4d3b-a2e3-85e47dc42996 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:00.736825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7955c58-f6b3-4d9d-a0d7-b464edf04b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:00.737436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7955c58-f6b3-4d9d-a0d7-b464edf04b69 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:00.737717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7955c58-f6b3-4d9d-a0d7-b464edf04b69 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:00.737990 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7955c58-f6b3-4d9d-a0d7-b464edf04b69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:00.739086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7955c58-f6b3-4d9d-a0d7-b464edf04b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:00.740540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7955c58-f6b3-4d9d-a0d7-b464edf04b69 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:00.742881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 425/863] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:00.748689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0663838-5980-4d44-b51d-5fcf13e665ea None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:00.839289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:00.842263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0663838-5980-4d44-b51d-5fcf13e665ea None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:00.876095 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56046955-c921-4cd4-bcb3-968f0a016291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:00.876654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56046955-c921-4cd4-bcb3-968f0a016291 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:00.876933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56046955-c921-4cd4-bcb3-968f0a016291 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:00.877202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56046955-c921-4cd4-bcb3-968f0a016291 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:00.877836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56046955-c921-4cd4-bcb3-968f0a016291 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:00.879241 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56046955-c921-4cd4-bcb3-968f0a016291 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:00.899046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 439/864] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:30:00 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:30:03.049720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a0e6d-9ca7-4204-a3fb-32723d726a58 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:03.050297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a0e6d-9ca7-4204-a3fb-32723d726a58 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:03.050297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a0e6d-9ca7-4204-a3fb-32723d726a58 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:03.051590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 426/865] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:30:03.057221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548ff96-41ed-42f1-97d3-8df7c3102f84 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:03.057638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548ff96-41ed-42f1-97d3-8df7c3102f84 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:03.057773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548ff96-41ed-42f1-97d3-8df7c3102f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:03.075793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0548ff96-41ed-42f1-97d3-8df7c3102f84 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:30:03.076980 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0548ff96-41ed-42f1-97d3-8df7c3102f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:30:03.139870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 440/866] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:30:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:30:03.416494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a0e6d-9ca7-4204-a3fb-32723d726a58 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:03.505887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:03.516582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a0e6d-9ca7-4204-a3fb-32723d726a58 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:03.549753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fc227-61f9-4242-a3d6-b00dc8a90762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:03.550368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fc227-61f9-4242-a3d6-b00dc8a90762 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:03.550647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fc227-61f9-4242-a3d6-b00dc8a90762 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:03.550921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fc227-61f9-4242-a3d6-b00dc8a90762 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:03.560726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fc227-61f9-4242-a3d6-b00dc8a90762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-c58fc227-61f9-4242-a3d6-b00dc8a90762 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:30:03.565463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:30:03.566600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:30:03.567837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 427/867] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:30:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:30:03.572694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548ff96-41ed-42f1-97d3-8df7c3102f84 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:03.652420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:03.667459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548ff96-41ed-42f1-97d3-8df7c3102f84 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:03.699361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf0d89-9848-4698-92af-4622889107bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:03.699955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf0d89-9848-4698-92af-4622889107bf None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:03.700236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf0d89-9848-4698-92af-4622889107bf None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:03.700541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf0d89-9848-4698-92af-4622889107bf None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:03.701200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf0d89-9848-4698-92af-4622889107bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:30:03.701505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf0d89-9848-4698-92af-4622889107bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:03.702864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf0d89-9848-4698-92af-4622889107bf None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:03.710581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 441/868] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:30:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:03.717519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fc227-61f9-4242-a3d6-b00dc8a90762 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:03.812833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fc227-61f9-4242-a3d6-b00dc8a90762 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:03.844940 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d472f6-0619-4ed5-9a2a-4f152f49b6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:03.845549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d472f6-0619-4ed5-9a2a-4f152f49b6af None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:03.845827 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d472f6-0619-4ed5-9a2a-4f152f49b6af None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:03.846098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d472f6-0619-4ed5-9a2a-4f152f49b6af None admin] PATH_INFO: `/v3/users/db160f5918454a55846e2abc49190b4f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:03.859206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d472f6-0619-4ed5-9a2a-4f152f49b6af None admin] RBAC: Authorizing `identity:get_user(user_id=db160f5918454a55846e2abc49190b4f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:03.860666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d472f6-0619-4ed5-9a2a-4f152f49b6af None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:03.862212 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 428/869] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:30:03 2022] GET /identity/v3/users/db160f5918454a55846e2abc49190b4f => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:03.868131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf0d89-9848-4698-92af-4622889107bf None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:03.963655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf0d89-9848-4698-92af-4622889107bf None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:03.995171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00158b25-4485-4aac-8f8d-fde73315789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:03.995771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00158b25-4485-4aac-8f8d-fde73315789c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:03.996071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00158b25-4485-4aac-8f8d-fde73315789c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:03.996374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00158b25-4485-4aac-8f8d-fde73315789c None admin] PATH_INFO: `/v3/projects/280c70b733744f329c768fd72d42eb27` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:03.997448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00158b25-4485-4aac-8f8d-fde73315789c None admin] RBAC: Authorizing `identity:get_project(project_id=280c70b733744f329c768fd72d42eb27)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:03.998854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00158b25-4485-4aac-8f8d-fde73315789c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:04.000434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 442/870] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:30:03 2022] GET /identity/v3/projects/280c70b733744f329c768fd72d42eb27 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:04.006145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d472f6-0619-4ed5-9a2a-4f152f49b6af None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:04.017075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:04.102228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d472f6-0619-4ed5-9a2a-4f152f49b6af None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:04.135569 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f796d7-b8f0-46d0-bf59-a152f78f2830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:04.136209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f796d7-b8f0-46d0-bf59-a152f78f2830 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:04.136522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f796d7-b8f0-46d0-bf59-a152f78f2830 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:04.136796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f796d7-b8f0-46d0-bf59-a152f78f2830 None admin] PATH_INFO: `/v3/projects/280c70b733744f329c768fd72d42eb27/users/db160f5918454a55846e2abc49190b4f/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:04.138644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f796d7-b8f0-46d0-bf59-a152f78f2830 None admin] RBAC: Authorizing `identity:create_grant(project_id=280c70b733744f329c768fd72d42eb27, user_id=db160f5918454a55846e2abc49190b4f, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:04.140062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f796d7-b8f0-46d0-bf59-a152f78f2830 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:04.154518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 429/871] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:30:04 2022] PUT /identity/v3/projects/280c70b733744f329c768fd72d42eb27/users/db160f5918454a55846e2abc49190b4f/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:30:06.283740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2a817-a41c-4475-821b-5847525b5a69 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:06.284294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2a817-a41c-4475-821b-5847525b5a69 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:06.284294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2a817-a41c-4475-821b-5847525b5a69 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:06.285740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 443/872] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:30:06.292106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac31aa-04b0-4bde-86de-6eb0f6acfaf2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:06.292550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac31aa-04b0-4bde-86de-6eb0f6acfaf2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:06.292679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac31aa-04b0-4bde-86de-6eb0f6acfaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:06.310998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-bdac31aa-04b0-4bde-86de-6eb0f6acfaf2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:30:06.312161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bdac31aa-04b0-4bde-86de-6eb0f6acfaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:30:06.372471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:06.378667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 430/873] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:30:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:30:06.647837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2a817-a41c-4475-821b-5847525b5a69 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:06.667046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:06.746218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2a817-a41c-4475-821b-5847525b5a69 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:06.777964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fef953-3c2d-4d68-bbf2-c0044a183426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:06.778572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fef953-3c2d-4d68-bbf2-c0044a183426 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:06.778850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fef953-3c2d-4d68-bbf2-c0044a183426 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:06.779121 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fef953-3c2d-4d68-bbf2-c0044a183426 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:06.788732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fef953-3c2d-4d68-bbf2-c0044a183426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-11fef953-3c2d-4d68-bbf2-c0044a183426 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:30:06.793585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:30:06.794629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:30:06.795471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 444/874] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:30:06.800559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac31aa-04b0-4bde-86de-6eb0f6acfaf2 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:06.895656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac31aa-04b0-4bde-86de-6eb0f6acfaf2 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:06.928004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1a135-cabc-49e7-b307-e5052d9e8c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:06.928569 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1a135-cabc-49e7-b307-e5052d9e8c6b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:06.928852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1a135-cabc-49e7-b307-e5052d9e8c6b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:06.929126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1a135-cabc-49e7-b307-e5052d9e8c6b None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:06.929862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1a135-cabc-49e7-b307-e5052d9e8c6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:30:06.930169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1a135-cabc-49e7-b307-e5052d9e8c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:06.931530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1a135-cabc-49e7-b307-e5052d9e8c6b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:06.939166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 431/875] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:06.945591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fef953-3c2d-4d68-bbf2-c0044a183426 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:07.019276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:07.041814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fef953-3c2d-4d68-bbf2-c0044a183426 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:07.073568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311ed8c-7b2c-4559-9c73-d6299b270eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:07.074172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311ed8c-7b2c-4559-9c73-d6299b270eed None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:07.074450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311ed8c-7b2c-4559-9c73-d6299b270eed None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:07.074722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311ed8c-7b2c-4559-9c73-d6299b270eed None admin] PATH_INFO: `/v3/users/db160f5918454a55846e2abc49190b4f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:07.075810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311ed8c-7b2c-4559-9c73-d6299b270eed None admin] RBAC: Authorizing `identity:get_user(user_id=db160f5918454a55846e2abc49190b4f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:07.077254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311ed8c-7b2c-4559-9c73-d6299b270eed None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:07.078778 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 445/876] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:30:06 2022] GET /identity/v3/users/db160f5918454a55846e2abc49190b4f => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:30:07.084528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1a135-cabc-49e7-b307-e5052d9e8c6b None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:07.142194 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:07.182463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1a135-cabc-49e7-b307-e5052d9e8c6b None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:07.215374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34845407-43b2-47f5-ba74-83eab09426c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:07.215965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34845407-43b2-47f5-ba74-83eab09426c7 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:07.216246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34845407-43b2-47f5-ba74-83eab09426c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:07.216551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34845407-43b2-47f5-ba74-83eab09426c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:07.281857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34845407-43b2-47f5-ba74-83eab09426c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:07.283538 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34845407-43b2-47f5-ba74-83eab09426c7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:07.339339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:07.380515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 432/877] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:30:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 20 14:30:07.386969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311ed8c-7b2c-4559-9c73-d6299b270eed None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:07.479849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311ed8c-7b2c-4559-9c73-d6299b270eed None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:07.512008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8dff1a-811e-4c54-87cc-a52291e68210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:07.512643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dff1a-811e-4c54-87cc-a52291e68210 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:07.512919 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dff1a-811e-4c54-87cc-a52291e68210 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:07.513187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dff1a-811e-4c54-87cc-a52291e68210 None admin] PATH_INFO: `/v3/projects/280c70b733744f329c768fd72d42eb27` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:07.514265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dff1a-811e-4c54-87cc-a52291e68210 None admin] RBAC: Authorizing `identity:get_project(project_id=280c70b733744f329c768fd72d42eb27)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:07.515671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dff1a-811e-4c54-87cc-a52291e68210 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:07.517266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 446/878] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:30:07 2022] GET /identity/v3/projects/280c70b733744f329c768fd72d42eb27 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:07.524250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34845407-43b2-47f5-ba74-83eab09426c7 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:07.620877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34845407-43b2-47f5-ba74-83eab09426c7 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:07.653805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df81f6-74c2-45a5-a27c-7ee6a0d1c622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:07.654391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df81f6-74c2-45a5-a27c-7ee6a0d1c622 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:07.654668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df81f6-74c2-45a5-a27c-7ee6a0d1c622 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:07.654938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df81f6-74c2-45a5-a27c-7ee6a0d1c622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:07.655648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df81f6-74c2-45a5-a27c-7ee6a0d1c622 None admin] RBAC: Adding query filter params (role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.project.id=280c70b733744f329c768fd72d42eb27, user.id=db160f5918454a55846e2abc49190b4f) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:30:07.655950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df81f6-74c2-45a5-a27c-7ee6a0d1c622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:07.657406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df81f6-74c2-45a5-a27c-7ee6a0d1c622 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:07.662849 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 433/879] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:30:07 2022] GET /identity/v3/role_assignments?user.id=db160f5918454a55846e2abc49190b4f&scope.project.id=280c70b733744f329c768fd72d42eb27&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:09.803600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5704d-d547-4ef4-8de6-9ec2d86d30e7 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:09.804120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5704d-d547-4ef4-8de6-9ec2d86d30e7 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:09.804120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5704d-d547-4ef4-8de6-9ec2d86d30e7 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:09.806260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 447/880] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:30:09.811466 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7496e-258d-4f63-aa9b-b22d1d36f09c None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:09.811884 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7496e-258d-4f63-aa9b-b22d1d36f09c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:09.812004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7496e-258d-4f63-aa9b-b22d1d36f09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:09.830129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-95d7496e-258d-4f63-aa9b-b22d1d36f09c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:30:09.831301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-95d7496e-258d-4f63-aa9b-b22d1d36f09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:30:09.847792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:09.896834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 434/881] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:30:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:30:10.170217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5704d-d547-4ef4-8de6-9ec2d86d30e7 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:10.189185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:10.270926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5704d-d547-4ef4-8de6-9ec2d86d30e7 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:10.303621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1769e-a5f4-416e-8b95-6800798a59fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:10.304710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1769e-a5f4-416e-8b95-6800798a59fd None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:10.304996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1769e-a5f4-416e-8b95-6800798a59fd None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:10.305312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1769e-a5f4-416e-8b95-6800798a59fd None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:10.305932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1769e-a5f4-416e-8b95-6800798a59fd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:10.307290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1769e-a5f4-416e-8b95-6800798a59fd None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:10.322884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 448/882] 10.222.0.13 () {60 vars in 1138 bytes} [Thu Oct 20 14:30:10 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:30:12.485679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625e2c1-cba1-4d91-a9ec-fe3ac803fe1d None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:12.486223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625e2c1-cba1-4d91-a9ec-fe3ac803fe1d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:12.486223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625e2c1-cba1-4d91-a9ec-fe3ac803fe1d None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:12.487548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 435/883] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:30:12.493765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5dcbf-c297-4653-9bf5-b5b036bec4c8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:12.494176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5dcbf-c297-4653-9bf5-b5b036bec4c8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:12.494320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5dcbf-c297-4653-9bf5-b5b036bec4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:12.513069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e9a5dcbf-c297-4653-9bf5-b5b036bec4c8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:30:12.514257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e9a5dcbf-c297-4653-9bf5-b5b036bec4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:30:12.578648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 449/884] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:30:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:30:12.846669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625e2c1-cba1-4d91-a9ec-fe3ac803fe1d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:12.946604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625e2c1-cba1-4d91-a9ec-fe3ac803fe1d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:12.981887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95c1f5-b509-4630-88e2-ddcc8eb65561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:12.982509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95c1f5-b509-4630-88e2-ddcc8eb65561 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:12.982792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95c1f5-b509-4630-88e2-ddcc8eb65561 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:12.983068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95c1f5-b509-4630-88e2-ddcc8eb65561 None admin] PATH_INFO: `/v3/domains/ee2bb0d154a146a6aaef5309720aeb5e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:12.993423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95c1f5-b509-4630-88e2-ddcc8eb65561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee2bb0d154a146a6aaef5309720aeb5e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:12.994859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95c1f5-b509-4630-88e2-ddcc8eb65561 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:12.996447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 436/885] 10.222.0.13 () {56 vars in 1183 bytes} [Thu Oct 20 14:30:12 2022] GET /identity/v3/domains/ee2bb0d154a146a6aaef5309720aeb5e => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:13.002315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5dcbf-c297-4653-9bf5-b5b036bec4c8 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:13.004451 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:13.100169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5dcbf-c297-4653-9bf5-b5b036bec4c8 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:13.131946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035b9bc-d7f6-4263-b403-76157090f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:13.133137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035b9bc-d7f6-4263-b403-76157090f7ca None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:13.133337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035b9bc-d7f6-4263-b403-76157090f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:13.133606 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035b9bc-d7f6-4263-b403-76157090f7ca None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:13.134242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035b9bc-d7f6-4263-b403-76157090f7ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:13.135661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035b9bc-d7f6-4263-b403-76157090f7ca None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:13.162429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 450/886] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:30:13 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:30:15.329303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f79fac-325e-40ae-8c73-2731bab72948 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:15.330016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f79fac-325e-40ae-8c73-2731bab72948 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:15.330016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f79fac-325e-40ae-8c73-2731bab72948 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:15.331157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 437/887] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:30:15.337408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0907d-e1c6-4001-9663-bb4048e37b55 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:15.337845 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0907d-e1c6-4001-9663-bb4048e37b55 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:15.337967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0907d-e1c6-4001-9663-bb4048e37b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:15.356693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-faf0907d-e1c6-4001-9663-bb4048e37b55 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:30:15.357872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-faf0907d-e1c6-4001-9663-bb4048e37b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:30:15.407318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:15.424025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 451/888] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:30:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:30:15.693243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f79fac-325e-40ae-8c73-2731bab72948 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:15.709407 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:15.795971 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f79fac-325e-40ae-8c73-2731bab72948 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:15.829297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e585b-3bab-43f3-afd5-44a296c3070d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:15.829922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e585b-3bab-43f3-afd5-44a296c3070d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:15.830203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e585b-3bab-43f3-afd5-44a296c3070d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:15.830485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e585b-3bab-43f3-afd5-44a296c3070d None admin] PATH_INFO: `/v3/domains/ee2bb0d154a146a6aaef5309720aeb5e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:15.831582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e585b-3bab-43f3-afd5-44a296c3070d None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee2bb0d154a146a6aaef5309720aeb5e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:15.833051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e585b-3bab-43f3-afd5-44a296c3070d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:15.834609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 438/889] 10.222.0.13 () {56 vars in 1183 bytes} [Thu Oct 20 14:30:15 2022] GET /identity/v3/domains/ee2bb0d154a146a6aaef5309720aeb5e => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:15.840988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0907d-e1c6-4001-9663-bb4048e37b55 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:15.939117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0907d-e1c6-4001-9663-bb4048e37b55 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:15.972663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc0839-b57b-4854-a853-bcf0be26c596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:15.973205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc0839-b57b-4854-a853-bcf0be26c596 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:15.973482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc0839-b57b-4854-a853-bcf0be26c596 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:15.973753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc0839-b57b-4854-a853-bcf0be26c596 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:15.974402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc0839-b57b-4854-a853-bcf0be26c596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:15.976386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc0839-b57b-4854-a853-bcf0be26c596 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:15.996956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 452/890] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:30:15 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:30:18.149342 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ba1f-ed61-4929-b76e-c9b60cc15254 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:18.149892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ba1f-ed61-4929-b76e-c9b60cc15254 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:18.149892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ba1f-ed61-4929-b76e-c9b60cc15254 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:18.151217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 439/891] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:30:18.156386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2ca5d-47cf-463c-b870-7b20c5b19153 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:18.156815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2ca5d-47cf-463c-b870-7b20c5b19153 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:18.156947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2ca5d-47cf-463c-b870-7b20c5b19153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:18.175555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d9f2ca5d-47cf-463c-b870-7b20c5b19153 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:30:18.176766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d9f2ca5d-47cf-463c-b870-7b20c5b19153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:30:18.211470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:18.246791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 453/892] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:30:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:30:18.514655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ba1f-ed61-4929-b76e-c9b60cc15254 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:18.595269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:18.614573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ba1f-ed61-4929-b76e-c9b60cc15254 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:18.646786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798924b-4efe-4d60-a652-dffebeac80f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:18.647410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798924b-4efe-4d60-a652-dffebeac80f7 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:18.648296 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798924b-4efe-4d60-a652-dffebeac80f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:18.648676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798924b-4efe-4d60-a652-dffebeac80f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:18.658819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798924b-4efe-4d60-a652-dffebeac80f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-5798924b-4efe-4d60-a652-dffebeac80f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:30:18.664201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:30:18.665384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:30:18.666293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 440/893] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:30:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:30:18.671362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2ca5d-47cf-463c-b870-7b20c5b19153 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:18.764029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2ca5d-47cf-463c-b870-7b20c5b19153 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:18.797281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e04810-c090-4c48-94b9-bec48f6f687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:18.797825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e04810-c090-4c48-94b9-bec48f6f687d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:18.798103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e04810-c090-4c48-94b9-bec48f6f687d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:18.798373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e04810-c090-4c48-94b9-bec48f6f687d None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:18.799023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e04810-c090-4c48-94b9-bec48f6f687d None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:30:18.799322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e04810-c090-4c48-94b9-bec48f6f687d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:18.800719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e04810-c090-4c48-94b9-bec48f6f687d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:18.808580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 454/894] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:30:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:18.814413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798924b-4efe-4d60-a652-dffebeac80f7 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:18.911578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798924b-4efe-4d60-a652-dffebeac80f7 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:18.943083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94957fb0-cf88-456e-bb23-daab7cf4829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:18.943691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94957fb0-cf88-456e-bb23-daab7cf4829c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:18.943963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94957fb0-cf88-456e-bb23-daab7cf4829c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:18.944231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94957fb0-cf88-456e-bb23-daab7cf4829c None admin] PATH_INFO: `/v3/users/6ae29b567e9442afa89334742456f8e2` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:18.955647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:18.958728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94957fb0-cf88-456e-bb23-daab7cf4829c None admin] RBAC: Authorizing `identity:get_user(user_id=6ae29b567e9442afa89334742456f8e2)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:18.960566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94957fb0-cf88-456e-bb23-daab7cf4829c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:18.962355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 441/895] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:30:18 2022] GET /identity/v3/users/6ae29b567e9442afa89334742456f8e2 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:18.967979 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e04810-c090-4c48-94b9-bec48f6f687d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:18.999816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:19.065774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e04810-c090-4c48-94b9-bec48f6f687d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:19.097288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f1a65-c71f-4c2d-9d52-808fce5f9dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:19.097885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f1a65-c71f-4c2d-9d52-808fce5f9dcd None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:19.098157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f1a65-c71f-4c2d-9d52-808fce5f9dcd None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:19.098426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f1a65-c71f-4c2d-9d52-808fce5f9dcd None admin] PATH_INFO: `/v3/projects/214b0adf07054ec8914d286c54228582` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:19.099506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f1a65-c71f-4c2d-9d52-808fce5f9dcd None admin] RBAC: Authorizing `identity:get_project(project_id=214b0adf07054ec8914d286c54228582)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:19.101474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f1a65-c71f-4c2d-9d52-808fce5f9dcd None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:19.103059 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 455/896] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:30:18 2022] GET /identity/v3/projects/214b0adf07054ec8914d286c54228582 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:19.109150 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94957fb0-cf88-456e-bb23-daab7cf4829c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:19.204953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94957fb0-cf88-456e-bb23-daab7cf4829c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:19.238625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e4588-0739-4a87-be50-8e3a14486ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:19.239272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e4588-0739-4a87-be50-8e3a14486ec4 None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:19.239551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e4588-0739-4a87-be50-8e3a14486ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:19.239825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e4588-0739-4a87-be50-8e3a14486ec4 None admin] PATH_INFO: `/v3/projects/214b0adf07054ec8914d286c54228582/users/6ae29b567e9442afa89334742456f8e2/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:19.241991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e4588-0739-4a87-be50-8e3a14486ec4 None admin] RBAC: Authorizing `identity:create_grant(project_id=214b0adf07054ec8914d286c54228582, user_id=6ae29b567e9442afa89334742456f8e2, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:19.243440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e4588-0739-4a87-be50-8e3a14486ec4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:19.255103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 442/897] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:30:19 2022] PUT /identity/v3/projects/214b0adf07054ec8914d286c54228582/users/6ae29b567e9442afa89334742456f8e2/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:30:21.369099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ead850-246b-4e34-87af-471a66b56af3 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:21.369501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ead850-246b-4e34-87af-471a66b56af3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:21.369645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ead850-246b-4e34-87af-471a66b56af3 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:21.370970 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 456/898] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:30:21.376039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab130a62-462e-455b-baa3-622bca91072f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:21.376491 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab130a62-462e-455b-baa3-622bca91072f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:21.376590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab130a62-462e-455b-baa3-622bca91072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:21.394242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ab130a62-462e-455b-baa3-622bca91072f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:30:21.395397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ab130a62-462e-455b-baa3-622bca91072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:30:21.448803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:21.461044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 443/899] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:30:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:30:21.726064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ead850-246b-4e34-87af-471a66b56af3 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:21.819068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:21.823605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ead850-246b-4e34-87af-471a66b56af3 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:21.857543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373af48e-5c1f-46bc-9754-f134b1c35771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:21.858162 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373af48e-5c1f-46bc-9754-f134b1c35771 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:21.858443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373af48e-5c1f-46bc-9754-f134b1c35771 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:21.858715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373af48e-5c1f-46bc-9754-f134b1c35771 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:21.868928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373af48e-5c1f-46bc-9754-f134b1c35771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-373af48e-5c1f-46bc-9754-f134b1c35771 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:30:21.873694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:30:21.874764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:30:21.875769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 457/900] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:30:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:30:21.880993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab130a62-462e-455b-baa3-622bca91072f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:21.977574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab130a62-462e-455b-baa3-622bca91072f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:22.010103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c22613-6754-459b-b3b5-b0ea102e07ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:22.010691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c22613-6754-459b-b3b5-b0ea102e07ec None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:22.010965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c22613-6754-459b-b3b5-b0ea102e07ec None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:22.011232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c22613-6754-459b-b3b5-b0ea102e07ec None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:22.011879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c22613-6754-459b-b3b5-b0ea102e07ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:30:22.012179 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c22613-6754-459b-b3b5-b0ea102e07ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:22.013555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c22613-6754-459b-b3b5-b0ea102e07ec None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:22.021122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 444/901] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:30:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:22.027716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373af48e-5c1f-46bc-9754-f134b1c35771 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:22.121578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373af48e-5c1f-46bc-9754-f134b1c35771 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:22.153926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8984a-9cad-4b66-8ac8-4442e541231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:22.154537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8984a-9cad-4b66-8ac8-4442e541231f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:22.154810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8984a-9cad-4b66-8ac8-4442e541231f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:22.155082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8984a-9cad-4b66-8ac8-4442e541231f None admin] PATH_INFO: `/v3/users/6ae29b567e9442afa89334742456f8e2` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:22.156169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8984a-9cad-4b66-8ac8-4442e541231f None admin] RBAC: Authorizing `identity:get_user(user_id=6ae29b567e9442afa89334742456f8e2)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:22.157623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8984a-9cad-4b66-8ac8-4442e541231f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:22.159166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 458/902] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Oct 20 14:30:22 2022] GET /identity/v3/users/6ae29b567e9442afa89334742456f8e2 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:22.165446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c22613-6754-459b-b3b5-b0ea102e07ec None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:22.215828 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:22.262056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c22613-6754-459b-b3b5-b0ea102e07ec None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:22.295499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5468f5-b0f3-44f2-afec-6ba4119032d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:22.296098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5468f5-b0f3-44f2-afec-6ba4119032d3 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:22.296402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5468f5-b0f3-44f2-afec-6ba4119032d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:22.296676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5468f5-b0f3-44f2-afec-6ba4119032d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:22.363308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5468f5-b0f3-44f2-afec-6ba4119032d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:22.364804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5468f5-b0f3-44f2-afec-6ba4119032d3 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:22.416287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:22.462544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 445/903] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:30:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 14:30:22.468655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8984a-9cad-4b66-8ac8-4442e541231f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:22.495982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:22.565289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8984a-9cad-4b66-8ac8-4442e541231f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:22.597818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d3baa-1cd3-42d0-b7e9-fc27d0ec019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:22.598465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d3baa-1cd3-42d0-b7e9-fc27d0ec019c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:22.598746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d3baa-1cd3-42d0-b7e9-fc27d0ec019c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:22.599019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d3baa-1cd3-42d0-b7e9-fc27d0ec019c None admin] PATH_INFO: `/v3/projects/214b0adf07054ec8914d286c54228582` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:22.600112 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d3baa-1cd3-42d0-b7e9-fc27d0ec019c None admin] RBAC: Authorizing `identity:get_project(project_id=214b0adf07054ec8914d286c54228582)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:22.602214 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d3baa-1cd3-42d0-b7e9-fc27d0ec019c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:22.603822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 459/904] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Oct 20 14:30:22 2022] GET /identity/v3/projects/214b0adf07054ec8914d286c54228582 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:22.609964 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5468f5-b0f3-44f2-afec-6ba4119032d3 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:30:22.705409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5468f5-b0f3-44f2-afec-6ba4119032d3 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:30:22.738377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7731a-d479-44a8-a9f0-e50ddc7953bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:30:22.738972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7731a-d479-44a8-a9f0-e50ddc7953bd None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:30:22.739252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7731a-d479-44a8-a9f0-e50ddc7953bd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:30:22.739524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7731a-d479-44a8-a9f0-e50ddc7953bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:30:22.740240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7731a-d479-44a8-a9f0-e50ddc7953bd None admin] RBAC: Adding query filter params (role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.project.id=214b0adf07054ec8914d286c54228582, user.id=6ae29b567e9442afa89334742456f8e2) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:30:22.740583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7731a-d479-44a8-a9f0-e50ddc7953bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:30:22.742285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7731a-d479-44a8-a9f0-e50ddc7953bd None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:30:22.747830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 446/905] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:30:22 2022] GET /identity/v3/role_assignments?user.id=6ae29b567e9442afa89334742456f8e2&scope.project.id=214b0adf07054ec8914d286c54228582&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:30:32.593777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=52 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:30:32.745398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:18.547647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bf53-ae94-4bdb-bd19-0f9ad72e53e2 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:18.547647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bf53-ae94-4bdb-bd19-0f9ad72e53e2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:18.547647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bf53-ae94-4bdb-bd19-0f9ad72e53e2 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:18.547647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 460/906] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:31:18.556380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166870f-39a5-49bf-b9c6-5a1bf7f17bc0 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:18.556380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166870f-39a5-49bf-b9c6-5a1bf7f17bc0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:18.556380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166870f-39a5-49bf-b9c6-5a1bf7f17bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:18.574691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-2166870f-39a5-49bf-b9c6-5a1bf7f17bc0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:31:18.575963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2166870f-39a5-49bf-b9c6-5a1bf7f17bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:31:18.664464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 447/907] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:31:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:31:18.961131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bf53-ae94-4bdb-bd19-0f9ad72e53e2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:19.101546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bf53-ae94-4bdb-bd19-0f9ad72e53e2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:19.145890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c2c5b-b6d1-4750-910f-fa4db5d8c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:19.146602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c2c5b-b6d1-4750-910f-fa4db5d8c1ad None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:19.146995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c2c5b-b6d1-4750-910f-fa4db5d8c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:19.147271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c2c5b-b6d1-4750-910f-fa4db5d8c1ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:19.160951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c2c5b-b6d1-4750-910f-fa4db5d8c1ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:19.162429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c2c5b-b6d1-4750-910f-fa4db5d8c1ad None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-710c2c5b-b6d1-4750-910f-fa4db5d8c1ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:31:19.181250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:19.182683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:31:19.183914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:31:19.183914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:19.183914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:19.183914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 461/908] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:19.189116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166870f-39a5-49bf-b9c6-5a1bf7f17bc0 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:19.317840 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166870f-39a5-49bf-b9c6-5a1bf7f17bc0 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:19.359830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:19.366961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0409f4-9d4d-4f44-9ff6-fcefe7585c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:19.367659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0409f4-9d4d-4f44-9ff6-fcefe7585c8c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:19.368235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0409f4-9d4d-4f44-9ff6-fcefe7585c8c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:19.368702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0409f4-9d4d-4f44-9ff6-fcefe7585c8c None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:19.369495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0409f4-9d4d-4f44-9ff6-fcefe7585c8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:19.369863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0409f4-9d4d-4f44-9ff6-fcefe7585c8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:19.371704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0409f4-9d4d-4f44-9ff6-fcefe7585c8c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:19.386057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 448/909] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:31:19.392943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c2c5b-b6d1-4750-910f-fa4db5d8c1ad None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:19.459003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:19.524401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c2c5b-b6d1-4750-910f-fa4db5d8c1ad None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:19.577051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ade8b5-c709-42bf-b95d-851d161d17dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:19.578334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ade8b5-c709-42bf-b95d-851d161d17dc None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:19.578777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ade8b5-c709-42bf-b95d-851d161d17dc None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:19.579217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ade8b5-c709-42bf-b95d-851d161d17dc None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:19.580027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ade8b5-c709-42bf-b95d-851d161d17dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:19.581727 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ade8b5-c709-42bf-b95d-851d161d17dc None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:19.609797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 462/910] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Oct 20 14:31:19 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:31:21.948360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6766-2cd2-471a-8831-737e083eb629 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:21.948943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6766-2cd2-471a-8831-737e083eb629 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:21.948943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6766-2cd2-471a-8831-737e083eb629 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:21.950250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 449/911] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:31:21.955785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5d145-1a12-4e29-bffc-e2b2961d2db6 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:21.956364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5d145-1a12-4e29-bffc-e2b2961d2db6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:21.956364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5d145-1a12-4e29-bffc-e2b2961d2db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:21.982418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-00c5d145-1a12-4e29-bffc-e2b2961d2db6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:31:21.983772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-00c5d145-1a12-4e29-bffc-e2b2961d2db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:31:22.069604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 463/912] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:31:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:31:22.385112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6766-2cd2-471a-8831-737e083eb629 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:22.510064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6766-2cd2-471a-8831-737e083eb629 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:22.549816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081ce1c-59c4-404a-ada4-94ccf99922a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:22.551953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ce1c-59c4-404a-ada4-94ccf99922a1 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:22.551953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ce1c-59c4-404a-ada4-94ccf99922a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:22.551953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ce1c-59c4-404a-ada4-94ccf99922a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:22.565404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081ce1c-59c4-404a-ada4-94ccf99922a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-2081ce1c-59c4-404a-ada4-94ccf99922a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:31:22.571758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:22.573004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:31:22.573994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:22.576421 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 450/913] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:31:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:22.583709 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5d145-1a12-4e29-bffc-e2b2961d2db6 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:22.633620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:22.705185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5d145-1a12-4e29-bffc-e2b2961d2db6 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:22.747038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a2bb8-1998-422b-812e-bd7b31f5f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:22.747705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a2bb8-1998-422b-812e-bd7b31f5f69f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:22.750153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a2bb8-1998-422b-812e-bd7b31f5f69f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:22.750153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a2bb8-1998-422b-812e-bd7b31f5f69f None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:22.750153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a2bb8-1998-422b-812e-bd7b31f5f69f None admin] RBAC: Adding query filter params (name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:22.750153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a2bb8-1998-422b-812e-bd7b31f5f69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:22.750598 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a2bb8-1998-422b-812e-bd7b31f5f69f None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:22.760407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 464/914] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:31:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:22.766531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081ce1c-59c4-404a-ada4-94ccf99922a1 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:22.829467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:22.886183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081ce1c-59c4-404a-ada4-94ccf99922a1 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:22.924525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e8979-ba43-447f-820e-38efaa36ce46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:22.925702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e8979-ba43-447f-820e-38efaa36ce46 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:22.925702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e8979-ba43-447f-820e-38efaa36ce46 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:22.925896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e8979-ba43-447f-820e-38efaa36ce46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:22.937120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e8979-ba43-447f-820e-38efaa36ce46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:22.938681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e8979-ba43-447f-820e-38efaa36ce46 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-410e8979-ba43-447f-820e-38efaa36ce46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:31:22.954660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:22.956335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:31:22.957601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:31:22.957601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:22.957601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:22.961492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 451/915] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:22.963103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a2bb8-1998-422b-812e-bd7b31f5f69f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:23.076248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:23.079407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a2bb8-1998-422b-812e-bd7b31f5f69f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:23.117972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aae627-d334-4571-b1e7-2014e901c147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:23.118651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aae627-d334-4571-b1e7-2014e901c147 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:23.119014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aae627-d334-4571-b1e7-2014e901c147 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:23.119214 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aae627-d334-4571-b1e7-2014e901c147 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:23.119933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aae627-d334-4571-b1e7-2014e901c147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:23.120234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aae627-d334-4571-b1e7-2014e901c147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:23.123263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aae627-d334-4571-b1e7-2014e901c147 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:23.143662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 465/916] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:31:23.144785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e8979-ba43-447f-820e-38efaa36ce46 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:23.262649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e8979-ba43-447f-820e-38efaa36ce46 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:23.264991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:23.303943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18edd75-9edd-4217-91e7-0e2e5b2c7550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:23.304615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18edd75-9edd-4217-91e7-0e2e5b2c7550 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:23.304958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18edd75-9edd-4217-91e7-0e2e5b2c7550 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:23.305234 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18edd75-9edd-4217-91e7-0e2e5b2c7550 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:23.313826 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18edd75-9edd-4217-91e7-0e2e5b2c7550 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:23.315611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18edd75-9edd-4217-91e7-0e2e5b2c7550 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-f18edd75-9edd-4217-91e7-0e2e5b2c7550 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:31:23.327622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:23.328908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 14:31:23.330173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:23.334924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 452/917] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:23.336867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aae627-d334-4571-b1e7-2014e901c147 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:23.452746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aae627-d334-4571-b1e7-2014e901c147 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:23.495008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b1d59-94c0-4dd7-870c-ba476a23f43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:23.495744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b1d59-94c0-4dd7-870c-ba476a23f43c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:23.496064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b1d59-94c0-4dd7-870c-ba476a23f43c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:23.496473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b1d59-94c0-4dd7-870c-ba476a23f43c None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:23.497267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b1d59-94c0-4dd7-870c-ba476a23f43c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:23.497665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b1d59-94c0-4dd7-870c-ba476a23f43c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:23.499371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b1d59-94c0-4dd7-870c-ba476a23f43c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:23.525749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 466/918] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Oct 20 14:31:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:31:23.532478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18edd75-9edd-4217-91e7-0e2e5b2c7550 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:23.679452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18edd75-9edd-4217-91e7-0e2e5b2c7550 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:23.715926 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aaab4c-cd47-4b2b-900b-b03c0a8b2f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:23.716504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aaab4c-cd47-4b2b-900b-b03c0a8b2f5d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:23.716837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aaab4c-cd47-4b2b-900b-b03c0a8b2f5d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:23.717117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aaab4c-cd47-4b2b-900b-b03c0a8b2f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:23.729108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aaab4c-cd47-4b2b-900b-b03c0a8b2f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:23.730766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aaab4c-cd47-4b2b-900b-b03c0a8b2f5d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-13aaab4c-cd47-4b2b-900b-b03c0a8b2f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:31:23.746730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:23.747735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:31:23.748614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:31:23.748614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:23.748614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:23.748614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 453/919] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:31:23.756376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b1d59-94c0-4dd7-870c-ba476a23f43c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:23.777592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:23.872690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b1d59-94c0-4dd7-870c-ba476a23f43c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:23.907799 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614f408-6588-4812-a878-9405428d72e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:23.908449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614f408-6588-4812-a878-9405428d72e5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:23.908759 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614f408-6588-4812-a878-9405428d72e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:23.909061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614f408-6588-4812-a878-9405428d72e5 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:23.909766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614f408-6588-4812-a878-9405428d72e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:23.910158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614f408-6588-4812-a878-9405428d72e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:23.911672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614f408-6588-4812-a878-9405428d72e5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:23.924747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 467/920] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:23.931028 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aaab4c-cd47-4b2b-900b-b03c0a8b2f5d None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:23.949559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:24.052978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aaab4c-cd47-4b2b-900b-b03c0a8b2f5d None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:24.089658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1eea2-5d70-47c3-8ef4-283b67b9ba27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:24.090283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1eea2-5d70-47c3-8ef4-283b67b9ba27 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:24.090612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1eea2-5d70-47c3-8ef4-283b67b9ba27 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:24.090883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1eea2-5d70-47c3-8ef4-283b67b9ba27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:24.097984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1eea2-5d70-47c3-8ef4-283b67b9ba27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:24.099457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1eea2-5d70-47c3-8ef4-283b67b9ba27 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-36b1eea2-5d70-47c3-8ef4-283b67b9ba27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:31:24.111188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:24.112113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:24.113334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:31:24.113334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:31:24.113334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:31:24.113334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:31:24.113334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:31:24.113334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:31:24.113334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:31:24.113334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:24.117734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 454/921] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:31:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:24.118767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614f408-6588-4812-a878-9405428d72e5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:24.205404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:24.234093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614f408-6588-4812-a878-9405428d72e5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:24.275190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed7585-05d8-4f05-b1df-b63fd1195df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:24.275809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed7585-05d8-4f05-b1df-b63fd1195df2 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:24.276093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed7585-05d8-4f05-b1df-b63fd1195df2 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:24.276381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed7585-05d8-4f05-b1df-b63fd1195df2 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:24.277081 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed7585-05d8-4f05-b1df-b63fd1195df2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:24.277390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed7585-05d8-4f05-b1df-b63fd1195df2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:24.279099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed7585-05d8-4f05-b1df-b63fd1195df2 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:24.292614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 468/922] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:31:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:24.299506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1eea2-5d70-47c3-8ef4-283b67b9ba27 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:24.399468 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:24.431756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1eea2-5d70-47c3-8ef4-283b67b9ba27 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:24.470974 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce0c5-4231-4e06-bd0f-c512b2228bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:24.471698 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce0c5-4231-4e06-bd0f-c512b2228bfd None admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:24.471980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce0c5-4231-4e06-bd0f-c512b2228bfd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:24.472331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce0c5-4231-4e06-bd0f-c512b2228bfd None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/fc18a78229064e8d8ea4321c3c23c4ba/roles/770887c5c808476d8505bf8b8fc5f27a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:24.490504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce0c5-4231-4e06-bd0f-c512b2228bfd None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=fc18a78229064e8d8ea4321c3c23c4ba, role_id=770887c5c808476d8505bf8b8fc5f27a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:24.492082 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce0c5-4231-4e06-bd0f-c512b2228bfd None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:24.508909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 455/923] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:31:24 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/fc18a78229064e8d8ea4321c3c23c4ba/roles/770887c5c808476d8505bf8b8fc5f27a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:31:26.854956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4944d-038f-4b87-a2b9-0bb48459bd42 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:26.855524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4944d-038f-4b87-a2b9-0bb48459bd42 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:26.855524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4944d-038f-4b87-a2b9-0bb48459bd42 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:26.856907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 469/924] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:31:26.862635 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13974e7d-7481-448d-89ce-18b37d7eac30 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:26.864119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13974e7d-7481-448d-89ce-18b37d7eac30 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:26.864119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13974e7d-7481-448d-89ce-18b37d7eac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:26.884263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-13974e7d-7481-448d-89ce-18b37d7eac30 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:31:26.885560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-13974e7d-7481-448d-89ce-18b37d7eac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:31:26.959815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 456/925] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:31:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:31:27.233278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4944d-038f-4b87-a2b9-0bb48459bd42 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:27.346868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4944d-038f-4b87-a2b9-0bb48459bd42 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:27.387888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569c53e-aac9-48a4-a45c-9521aded232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:27.388659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569c53e-aac9-48a4-a45c-9521aded232f None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:27.389038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569c53e-aac9-48a4-a45c-9521aded232f None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:27.389359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569c53e-aac9-48a4-a45c-9521aded232f None admin] PATH_INFO: `/v3/roles/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:27.396599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:27.401897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569c53e-aac9-48a4-a45c-9521aded232f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-4569c53e-aac9-48a4-a45c-9521aded232f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:31:27.407258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:27.408334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:27.409429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 470/926] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Oct 20 14:31:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:27.415511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13974e7d-7481-448d-89ce-18b37d7eac30 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:27.471265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:27.520822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13974e7d-7481-448d-89ce-18b37d7eac30 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:27.555253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62ba84-3a93-4bb8-8951-052589b4c8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:27.555838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62ba84-3a93-4bb8-8951-052589b4c8f9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:27.556142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62ba84-3a93-4bb8-8951-052589b4c8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:27.556426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62ba84-3a93-4bb8-8951-052589b4c8f9 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:27.557109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62ba84-3a93-4bb8-8951-052589b4c8f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:27.557396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62ba84-3a93-4bb8-8951-052589b4c8f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:27.558806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62ba84-3a93-4bb8-8951-052589b4c8f9 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:27.570651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 457/927] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Oct 20 14:31:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:27.576972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569c53e-aac9-48a4-a45c-9521aded232f None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:27.692361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569c53e-aac9-48a4-a45c-9521aded232f None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:27.729201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d73c7a-053e-41e9-9526-5de4f9642811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:27.729912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d73c7a-053e-41e9-9526-5de4f9642811 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:27.730193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d73c7a-053e-41e9-9526-5de4f9642811 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:27.730479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d73c7a-053e-41e9-9526-5de4f9642811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:27.741304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d73c7a-053e-41e9-9526-5de4f9642811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:27.742754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d73c7a-053e-41e9-9526-5de4f9642811 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-f2d73c7a-053e-41e9-9526-5de4f9642811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:31:27.759683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:27.760838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:31:27.761859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:31:27.761859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:27.761859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:27.761859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 471/928] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:27.766879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62ba84-3a93-4bb8-8951-052589b4c8f9 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:27.880909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62ba84-3a93-4bb8-8951-052589b4c8f9 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:27.899571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:27.919754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df933b-ccb9-4ffb-991b-e6334c6ae2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:27.920424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df933b-ccb9-4ffb-991b-e6334c6ae2f4 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:27.920707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df933b-ccb9-4ffb-991b-e6334c6ae2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:27.921001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df933b-ccb9-4ffb-991b-e6334c6ae2f4 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:27.921770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df933b-ccb9-4ffb-991b-e6334c6ae2f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:27.922022 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df933b-ccb9-4ffb-991b-e6334c6ae2f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:27.923488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df933b-ccb9-4ffb-991b-e6334c6ae2f4 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:27.936862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 458/929] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:27.942669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d73c7a-053e-41e9-9526-5de4f9642811 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:28.003942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:28.059345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d73c7a-053e-41e9-9526-5de4f9642811 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:28.101261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37726704-ab0a-4a4f-9c3d-3d676c5eff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:28.101890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37726704-ab0a-4a4f-9c3d-3d676c5eff65 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:28.102171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37726704-ab0a-4a4f-9c3d-3d676c5eff65 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:28.102445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37726704-ab0a-4a4f-9c3d-3d676c5eff65 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:28.110021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37726704-ab0a-4a4f-9c3d-3d676c5eff65 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:28.111492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37726704-ab0a-4a4f-9c3d-3d676c5eff65 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-37726704-ab0a-4a4f-9c3d-3d676c5eff65 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:31:28.124406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:28.125392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:28.126918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 472/930] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:31:28.132600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df933b-ccb9-4ffb-991b-e6334c6ae2f4 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:28.242473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df933b-ccb9-4ffb-991b-e6334c6ae2f4 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:28.278986 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d02dc3-583a-4793-9c26-71dc8ec30f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:28.279592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d02dc3-583a-4793-9c26-71dc8ec30f8d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:28.279870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d02dc3-583a-4793-9c26-71dc8ec30f8d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:28.280143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d02dc3-583a-4793-9c26-71dc8ec30f8d None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:28.280898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d02dc3-583a-4793-9c26-71dc8ec30f8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:28.281205 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d02dc3-583a-4793-9c26-71dc8ec30f8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:28.282650 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d02dc3-583a-4793-9c26-71dc8ec30f8d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:28.300887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 459/931] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Oct 20 14:31:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:28.306659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37726704-ab0a-4a4f-9c3d-3d676c5eff65 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:28.424995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37726704-ab0a-4a4f-9c3d-3d676c5eff65 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:28.436521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:28.465174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13378bbe-7f32-4d6e-bc83-c6c73cb514b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:28.465781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13378bbe-7f32-4d6e-bc83-c6c73cb514b0 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:28.466202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13378bbe-7f32-4d6e-bc83-c6c73cb514b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:28.466482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13378bbe-7f32-4d6e-bc83-c6c73cb514b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:28.542822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13378bbe-7f32-4d6e-bc83-c6c73cb514b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:28.544398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13378bbe-7f32-4d6e-bc83-c6c73cb514b0 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:28.655663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 473/932] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:31:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 14:31:28.661758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d02dc3-583a-4793-9c26-71dc8ec30f8d None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:28.700824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:28.772778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d02dc3-583a-4793-9c26-71dc8ec30f8d None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:28.811703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda25836-cdd5-456e-b96d-10fc49c33ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:28.812395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda25836-cdd5-456e-b96d-10fc49c33ecb None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:28.812718 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda25836-cdd5-456e-b96d-10fc49c33ecb None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:28.813086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda25836-cdd5-456e-b96d-10fc49c33ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:28.825084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda25836-cdd5-456e-b96d-10fc49c33ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:28.826640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda25836-cdd5-456e-b96d-10fc49c33ecb None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-dda25836-cdd5-456e-b96d-10fc49c33ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:31:28.841106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:28.842180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:31:28.843047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:31:28.843047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:28.843047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:28.846863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 460/933] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:28.848460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13378bbe-7f32-4d6e-bc83-c6c73cb514b0 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:28.859952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:28.953073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13378bbe-7f32-4d6e-bc83-c6c73cb514b0 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:28.992724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e331d8f-e493-44e2-b20f-c6cefaea3221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:28.993448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e331d8f-e493-44e2-b20f-c6cefaea3221 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:28.993724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e331d8f-e493-44e2-b20f-c6cefaea3221 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:28.994013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e331d8f-e493-44e2-b20f-c6cefaea3221 None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:28.994641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e331d8f-e493-44e2-b20f-c6cefaea3221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:28.994943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e331d8f-e493-44e2-b20f-c6cefaea3221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:28.996325 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e331d8f-e493-44e2-b20f-c6cefaea3221 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:29.009573 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 474/934] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:31:29.016077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda25836-cdd5-456e-b96d-10fc49c33ecb None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:29.110274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:29.127372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda25836-cdd5-456e-b96d-10fc49c33ecb None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:29.164261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a61c25-5f26-4c2c-809f-465ae5fa91cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:29.164945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a61c25-5f26-4c2c-809f-465ae5fa91cd None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:29.165243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a61c25-5f26-4c2c-809f-465ae5fa91cd None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:29.165561 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a61c25-5f26-4c2c-809f-465ae5fa91cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:29.174160 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a61c25-5f26-4c2c-809f-465ae5fa91cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:29.175745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a61c25-5f26-4c2c-809f-465ae5fa91cd None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-e5a61c25-5f26-4c2c-809f-465ae5fa91cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:31:29.188168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:29.189379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:29.190462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:31:29.190462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:31:29.190462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:31:29.190462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:31:29.190462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:31:29.190462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:31:29.190462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:31:29.190462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:29.190462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 461/935] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:31:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:29.197488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e331d8f-e493-44e2-b20f-c6cefaea3221 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:29.276741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:29.308696 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e331d8f-e493-44e2-b20f-c6cefaea3221 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:29.346287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277ef16-3066-4cdc-ab95-8260b21cf162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:29.347008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277ef16-3066-4cdc-ab95-8260b21cf162 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:29.348067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277ef16-3066-4cdc-ab95-8260b21cf162 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:29.348525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277ef16-3066-4cdc-ab95-8260b21cf162 None admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:29.349345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277ef16-3066-4cdc-ab95-8260b21cf162 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:29.349788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277ef16-3066-4cdc-ab95-8260b21cf162 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:29.351418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277ef16-3066-4cdc-ab95-8260b21cf162 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:29.365880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 475/936] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:31:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:29.372219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a61c25-5f26-4c2c-809f-465ae5fa91cd None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:29.479302 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a61c25-5f26-4c2c-809f-465ae5fa91cd None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:29.520261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702ff63-6b31-46b0-a015-93d8fe706247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:29.520889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ff63-6b31-46b0-a015-93d8fe706247 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:29.521167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ff63-6b31-46b0-a015-93d8fe706247 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:29.521436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ff63-6b31-46b0-a015-93d8fe706247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:29.522177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702ff63-6b31-46b0-a015-93d8fe706247 None admin] RBAC: Adding query filter params (role.id=770887c5c808476d8505bf8b8fc5f27a, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=fc18a78229064e8d8ea4321c3c23c4ba) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:29.522482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702ff63-6b31-46b0-a015-93d8fe706247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:29.523888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702ff63-6b31-46b0-a015-93d8fe706247 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:29.530500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 462/937] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:31:29 2022] GET /identity/v3/role_assignments?user.id=fc18a78229064e8d8ea4321c3c23c4ba&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=770887c5c808476d8505bf8b8fc5f27a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:31.864688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c987e1-b463-492d-aef0-3d71de6d5884 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:31.865497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c987e1-b463-492d-aef0-3d71de6d5884 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:31.865497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c987e1-b463-492d-aef0-3d71de6d5884 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:31.866953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 476/938] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:31:31.872541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38848492-a739-4445-95f0-3385a2efde0e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:31.873003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38848492-a739-4445-95f0-3385a2efde0e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:31.873114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38848492-a739-4445-95f0-3385a2efde0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:31.893730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-38848492-a739-4445-95f0-3385a2efde0e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:31:31.895700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-38848492-a739-4445-95f0-3385a2efde0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:31:31.898757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:31.974911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 463/939] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:31:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:31:32.261131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c987e1-b463-492d-aef0-3d71de6d5884 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:32.381388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c987e1-b463-492d-aef0-3d71de6d5884 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:32.412741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:32.419685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad29e1-3358-4844-9a8a-e5e9a245dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:32.420331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad29e1-3358-4844-9a8a-e5e9a245dc55 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:32.420699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad29e1-3358-4844-9a8a-e5e9a245dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:32.420981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad29e1-3358-4844-9a8a-e5e9a245dc55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:32.432820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad29e1-3358-4844-9a8a-e5e9a245dc55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-12ad29e1-3358-4844-9a8a-e5e9a245dc55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:31:32.437807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:32.438888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:32.440185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 477/940] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:32.446030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38848492-a739-4445-95f0-3385a2efde0e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:32.564030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38848492-a739-4445-95f0-3385a2efde0e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:32.601759 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d19ba-a710-4b68-9ddf-9857b68c9e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:32.602570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d19ba-a710-4b68-9ddf-9857b68c9e37 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:32.602938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d19ba-a710-4b68-9ddf-9857b68c9e37 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:32.603291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d19ba-a710-4b68-9ddf-9857b68c9e37 None admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:32.604102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d19ba-a710-4b68-9ddf-9857b68c9e37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:32.604529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d19ba-a710-4b68-9ddf-9857b68c9e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:32.606216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d19ba-a710-4b68-9ddf-9857b68c9e37 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:32.616990 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 464/941] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:31:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:32.623693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad29e1-3358-4844-9a8a-e5e9a245dc55 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:32.733997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad29e1-3358-4844-9a8a-e5e9a245dc55 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:32.776075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375f977-f196-43a5-bc43-2dc8b23f09c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:32.776978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375f977-f196-43a5-bc43-2dc8b23f09c8 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:32.777247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375f977-f196-43a5-bc43-2dc8b23f09c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:32.777625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375f977-f196-43a5-bc43-2dc8b23f09c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:32.787968 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375f977-f196-43a5-bc43-2dc8b23f09c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:32.789617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375f977-f196-43a5-bc43-2dc8b23f09c8 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-2375f977-f196-43a5-bc43-2dc8b23f09c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:31:32.807806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:32.814833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:31:32.816012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:31:32.816012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:32.816012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:32.816012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 478/942] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:31:32.816248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d19ba-a710-4b68-9ddf-9857b68c9e37 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:32.818980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:32.927557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d19ba-a710-4b68-9ddf-9857b68c9e37 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:32.961478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fa80e-5234-4279-aa11-09121ad49ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:32.962094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fa80e-5234-4279-aa11-09121ad49ca7 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:32.962499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fa80e-5234-4279-aa11-09121ad49ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:32.962821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fa80e-5234-4279-aa11-09121ad49ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:32.963668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fa80e-5234-4279-aa11-09121ad49ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:32.963972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fa80e-5234-4279-aa11-09121ad49ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:32.965380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fa80e-5234-4279-aa11-09121ad49ca7 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:32.981081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 465/943] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:32.988673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375f977-f196-43a5-bc43-2dc8b23f09c8 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:33.043245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:33.108017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375f977-f196-43a5-bc43-2dc8b23f09c8 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:33.144874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f40eb8-b6e2-41cf-b5bf-799663952ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:33.145640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f40eb8-b6e2-41cf-b5bf-799663952ea5 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:33.145880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f40eb8-b6e2-41cf-b5bf-799663952ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:33.146174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f40eb8-b6e2-41cf-b5bf-799663952ea5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:33.154241 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f40eb8-b6e2-41cf-b5bf-799663952ea5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:33.155891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f40eb8-b6e2-41cf-b5bf-799663952ea5 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-68f40eb8-b6e2-41cf-b5bf-799663952ea5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:31:33.168159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:33.169305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:33.170298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 479/944] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:33.175899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fa80e-5234-4279-aa11-09121ad49ca7 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:33.244892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:33.289377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fa80e-5234-4279-aa11-09121ad49ca7 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:33.327775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35038aa-e2b3-4968-93bd-19181dedc216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:33.328416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35038aa-e2b3-4968-93bd-19181dedc216 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:33.328700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35038aa-e2b3-4968-93bd-19181dedc216 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:33.328972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35038aa-e2b3-4968-93bd-19181dedc216 None admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:33.329688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35038aa-e2b3-4968-93bd-19181dedc216 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:33.329998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35038aa-e2b3-4968-93bd-19181dedc216 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:33.331550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35038aa-e2b3-4968-93bd-19181dedc216 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:33.350564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 466/945] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Oct 20 14:31:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:33.355922 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f40eb8-b6e2-41cf-b5bf-799663952ea5 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:33.467178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f40eb8-b6e2-41cf-b5bf-799663952ea5 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:33.468910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:33.505897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e31990-78f9-459c-931b-001e6066003c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:33.506590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e31990-78f9-459c-931b-001e6066003c None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:33.506873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e31990-78f9-459c-931b-001e6066003c None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:33.507177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e31990-78f9-459c-931b-001e6066003c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:33.519309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e31990-78f9-459c-931b-001e6066003c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:33.520875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e31990-78f9-459c-931b-001e6066003c None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-02e31990-78f9-459c-931b-001e6066003c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:31:33.538251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:33.544541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:31:33.545450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:31:33.545450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:33.545450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:33.545450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 480/946] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:33.545687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35038aa-e2b3-4968-93bd-19181dedc216 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:33.656811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35038aa-e2b3-4968-93bd-19181dedc216 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:33.680082 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:33.696317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f17ee4-723f-4722-99bc-e728045b5894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:33.697024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f17ee4-723f-4722-99bc-e728045b5894 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:33.697314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f17ee4-723f-4722-99bc-e728045b5894 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:33.697638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f17ee4-723f-4722-99bc-e728045b5894 None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:33.698346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f17ee4-723f-4722-99bc-e728045b5894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:33.698656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f17ee4-723f-4722-99bc-e728045b5894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:33.700134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f17ee4-723f-4722-99bc-e728045b5894 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:33.713112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 467/947] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:31:33.720052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e31990-78f9-459c-931b-001e6066003c None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:33.830304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e31990-78f9-459c-931b-001e6066003c None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:33.868645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091160d-b4b5-49e7-a91f-e76f534c9742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:33.869271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091160d-b4b5-49e7-a91f-e76f534c9742 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:33.869549 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091160d-b4b5-49e7-a91f-e76f534c9742 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:33.869856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091160d-b4b5-49e7-a91f-e76f534c9742 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:33.876822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091160d-b4b5-49e7-a91f-e76f534c9742 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:33.878328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091160d-b4b5-49e7-a91f-e76f534c9742 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-1091160d-b4b5-49e7-a91f-e76f534c9742 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:31:33.891300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:33.892438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:33.899173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:31:33.899173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:31:33.899173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:31:33.899173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:31:33.899173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:31:33.899173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:31:33.899173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:31:33.899173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:33.899173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 481/948] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:31:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:31:33.899765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f17ee4-723f-4722-99bc-e728045b5894 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:34.023664 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f17ee4-723f-4722-99bc-e728045b5894 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:34.065047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09314915-55ca-4f5c-8317-3b96230600ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:34.066641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09314915-55ca-4f5c-8317-3b96230600ad None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:34.066966 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09314915-55ca-4f5c-8317-3b96230600ad None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:34.067246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09314915-55ca-4f5c-8317-3b96230600ad None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:34.067939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09314915-55ca-4f5c-8317-3b96230600ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:34.068250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09314915-55ca-4f5c-8317-3b96230600ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:34.069761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09314915-55ca-4f5c-8317-3b96230600ad None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:34.082745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 468/949] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:31:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:34.089005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091160d-b4b5-49e7-a91f-e76f534c9742 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:34.106895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:34.201227 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091160d-b4b5-49e7-a91f-e76f534c9742 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:34.239262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778f38a-ce1d-4c62-bc38-cad6ccb28c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:34.240404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778f38a-ce1d-4c62-bc38-cad6ccb28c3e None admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:34.240750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778f38a-ce1d-4c62-bc38-cad6ccb28c3e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:34.241109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778f38a-ce1d-4c62-bc38-cad6ccb28c3e None admin] PATH_INFO: `/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/fc18a78229064e8d8ea4321c3c23c4ba/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:34.243269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778f38a-ce1d-4c62-bc38-cad6ccb28c3e None admin] RBAC: Authorizing `identity:create_grant(project_id=e49ab75eb6fe499a901d76ddc3ff8027, user_id=fc18a78229064e8d8ea4321c3c23c4ba, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:34.244731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778f38a-ce1d-4c62-bc38-cad6ccb28c3e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:34.259200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 482/950] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Oct 20 14:31:34 2022] PUT /identity/v3/projects/e49ab75eb6fe499a901d76ddc3ff8027/users/fc18a78229064e8d8ea4321c3c23c4ba/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:31:36.496547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e989-562d-4b69-8420-e10f8f806d83 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:36.497254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e989-562d-4b69-8420-e10f8f806d83 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:36.497254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e989-562d-4b69-8420-e10f8f806d83 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:36.498668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 469/951] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:31:36.504570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bac5db-ba17-4a2d-b635-4b040ee9197c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:36.505003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bac5db-ba17-4a2d-b635-4b040ee9197c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:36.505110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bac5db-ba17-4a2d-b635-4b040ee9197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:36.523963 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-16bac5db-ba17-4a2d-b635-4b040ee9197c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:31:36.525128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-16bac5db-ba17-4a2d-b635-4b040ee9197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:31:36.592673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 483/952] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:31:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:31:36.872951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694e989-562d-4b69-8420-e10f8f806d83 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:36.925230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:36.992274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694e989-562d-4b69-8420-e10f8f806d83 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:37.038251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb87414-fae6-41bb-81b3-8dda64fa186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:37.039350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb87414-fae6-41bb-81b3-8dda64fa186e None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:37.039719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb87414-fae6-41bb-81b3-8dda64fa186e None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:37.040222 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb87414-fae6-41bb-81b3-8dda64fa186e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:37.054462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb87414-fae6-41bb-81b3-8dda64fa186e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-3bb87414-fae6-41bb-81b3-8dda64fa186e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 14:31:37.060559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:37.061606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:37.062792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 470/953] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Oct 20 14:31:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 14:31:37.068506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bac5db-ba17-4a2d-b635-4b040ee9197c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:37.071614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:37.172650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bac5db-ba17-4a2d-b635-4b040ee9197c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:37.211031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223158c4-3901-47a6-a5b4-ab97eca4f635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:37.211721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223158c4-3901-47a6-a5b4-ab97eca4f635 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:37.212027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223158c4-3901-47a6-a5b4-ab97eca4f635 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:37.212312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223158c4-3901-47a6-a5b4-ab97eca4f635 None admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:37.213097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223158c4-3901-47a6-a5b4-ab97eca4f635 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:37.213470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223158c4-3901-47a6-a5b4-ab97eca4f635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:37.214915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223158c4-3901-47a6-a5b4-ab97eca4f635 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:37.224706 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 484/954] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Oct 20 14:31:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:37.230797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb87414-fae6-41bb-81b3-8dda64fa186e None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:37.343369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:37.347476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb87414-fae6-41bb-81b3-8dda64fa186e None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:37.387665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87899990-9e87-4b06-a95d-536d96413692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:37.388293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87899990-9e87-4b06-a95d-536d96413692 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:37.388703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87899990-9e87-4b06-a95d-536d96413692 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:37.388989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87899990-9e87-4b06-a95d-536d96413692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:37.399332 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87899990-9e87-4b06-a95d-536d96413692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:37.400817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87899990-9e87-4b06-a95d-536d96413692 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-87899990-9e87-4b06-a95d-536d96413692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:31:37.416322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:37.417490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:31:37.418411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:31:37.418411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:37.418411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:37.418411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 471/955] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:37.423836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223158c4-3901-47a6-a5b4-ab97eca4f635 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:37.494985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:37.535150 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223158c4-3901-47a6-a5b4-ab97eca4f635 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:37.570187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73258a-228b-4a7a-94bf-01795d539d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:37.571002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73258a-228b-4a7a-94bf-01795d539d3e None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:37.571383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73258a-228b-4a7a-94bf-01795d539d3e None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:37.571809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73258a-228b-4a7a-94bf-01795d539d3e None admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:37.572682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73258a-228b-4a7a-94bf-01795d539d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:37.572959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73258a-228b-4a7a-94bf-01795d539d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:37.574458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73258a-228b-4a7a-94bf-01795d539d3e None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:37.588605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 485/956] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:37.594495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87899990-9e87-4b06-a95d-536d96413692 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:37.702697 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87899990-9e87-4b06-a95d-536d96413692 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:37.739484 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28257afb-524f-4878-a60e-035e44750e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:37.740518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28257afb-524f-4878-a60e-035e44750e0c None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:37.740931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28257afb-524f-4878-a60e-035e44750e0c None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:37.741242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28257afb-524f-4878-a60e-035e44750e0c None admin] PATH_INFO: `/v3/users/placement` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:37.749152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28257afb-524f-4878-a60e-035e44750e0c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:37.750594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28257afb-524f-4878-a60e-035e44750e0c None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-28257afb-524f-4878-a60e-035e44750e0c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 14:31:37.763162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:37.764163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:37.765120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 472/957] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:37.770684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73258a-228b-4a7a-94bf-01795d539d3e None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:37.886364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73258a-228b-4a7a-94bf-01795d539d3e None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:37.914661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:37.925514 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88058337-c3e4-4b79-afc4-581952430555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:37.926257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88058337-c3e4-4b79-afc4-581952430555 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:37.926532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88058337-c3e4-4b79-afc4-581952430555 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:37.926848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88058337-c3e4-4b79-afc4-581952430555 None admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:37.927677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88058337-c3e4-4b79-afc4-581952430555 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:37.928023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88058337-c3e4-4b79-afc4-581952430555 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:37.929925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88058337-c3e4-4b79-afc4-581952430555 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:37.955781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 486/958] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Oct 20 14:31:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:31:37.957829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28257afb-524f-4878-a60e-035e44750e0c None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:37.974565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:38.083588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28257afb-524f-4878-a60e-035e44750e0c None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:38.121405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612795d4-9421-4450-a634-7917f015da9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:38.122018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612795d4-9421-4450-a634-7917f015da9a None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:38.122351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612795d4-9421-4450-a634-7917f015da9a None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:38.122655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612795d4-9421-4450-a634-7917f015da9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:38.195380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612795d4-9421-4450-a634-7917f015da9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:38.196906 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612795d4-9421-4450-a634-7917f015da9a None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:38.212327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:38.303290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 473/959] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Oct 20 14:31:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 14:31:38.309265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88058337-c3e4-4b79-afc4-581952430555 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:38.413562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88058337-c3e4-4b79-afc4-581952430555 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:38.449395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448b4f1-c834-4938-a9e0-d73abb84f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:38.450042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448b4f1-c834-4938-a9e0-d73abb84f230 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:38.450320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448b4f1-c834-4938-a9e0-d73abb84f230 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:38.450593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448b4f1-c834-4938-a9e0-d73abb84f230 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:38.461710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448b4f1-c834-4938-a9e0-d73abb84f230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:38.463232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448b4f1-c834-4938-a9e0-d73abb84f230 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-2448b4f1-c834-4938-a9e0-d73abb84f230 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 14:31:38.478635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:38.479737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 14:31:38.480665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 14:31:38.480665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 14:31:38.480665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:38.480665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 487/960] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:38.485663 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612795d4-9421-4450-a634-7917f015da9a None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:38.601190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612795d4-9421-4450-a634-7917f015da9a None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:38.614358 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:38.635905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f7403-f31f-4f3a-ae71-cfacab960ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:38.636699 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f7403-f31f-4f3a-ae71-cfacab960ead None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:38.637200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f7403-f31f-4f3a-ae71-cfacab960ead None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:38.637382 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f7403-f31f-4f3a-ae71-cfacab960ead None admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:38.638030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f7403-f31f-4f3a-ae71-cfacab960ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:38.638331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f7403-f31f-4f3a-ae71-cfacab960ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:38.639702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f7403-f31f-4f3a-ae71-cfacab960ead None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:38.651425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 474/961] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Oct 20 14:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:38.657127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448b4f1-c834-4938-a9e0-d73abb84f230 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:38.694135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:38.761137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448b4f1-c834-4938-a9e0-d73abb84f230 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:38.795126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03fb7-8f23-4024-9892-bd79d6aef67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:38.795752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03fb7-8f23-4024-9892-bd79d6aef67d None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:38.796028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03fb7-8f23-4024-9892-bd79d6aef67d None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:38.796299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03fb7-8f23-4024-9892-bd79d6aef67d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:38.802743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb03fb7-8f23-4024-9892-bd79d6aef67d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:38.804169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb03fb7-8f23-4024-9892-bd79d6aef67d None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application [None req-0cb03fb7-8f23-4024-9892-bd79d6aef67d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 14:31:38.814985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:38.816546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:38.817632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 14:31:38.817632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 14:31:38.817632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 14:31:38.817632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 14:31:38.817632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 14:31:38.817632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 14:31:38.817632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 14:31:38.817632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: ERROR keystone.server.flask.application Oct 20 14:31:38.817632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 488/962] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Oct 20 14:31:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:38.824022 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f7403-f31f-4f3a-ae71-cfacab960ead None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:38.920214 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f7403-f31f-4f3a-ae71-cfacab960ead None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:38.952858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0953e-ad14-4420-8963-a70e6aaf6b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:38.953451 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0953e-ad14-4420-8963-a70e6aaf6b5b None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:38.953732 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0953e-ad14-4420-8963-a70e6aaf6b5b None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:38.954004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0953e-ad14-4420-8963-a70e6aaf6b5b None admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:38.954682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0953e-ad14-4420-8963-a70e6aaf6b5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:38.954986 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0953e-ad14-4420-8963-a70e6aaf6b5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:38.956639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0953e-ad14-4420-8963-a70e6aaf6b5b None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:38.968078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 475/963] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Oct 20 14:31:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:38.975444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03fb7-8f23-4024-9892-bd79d6aef67d None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:39.071549 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:39.077361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03fb7-8f23-4024-9892-bd79d6aef67d None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:39.109088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ba029-b904-4e90-95ca-efddfffbac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:39.109674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ba029-b904-4e90-95ca-efddfffbac1a None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:39.109949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ba029-b904-4e90-95ca-efddfffbac1a None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:39.110216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ba029-b904-4e90-95ca-efddfffbac1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:39.110918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ba029-b904-4e90-95ca-efddfffbac1a None admin] RBAC: Adding query filter params (role.id=d8edc38e326243f28c9b81911ac5ed8f, scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027, user.id=fc18a78229064e8d8ea4321c3c23c4ba) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:39.111221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ba029-b904-4e90-95ca-efddfffbac1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:39.112764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ba029-b904-4e90-95ca-efddfffbac1a None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:39.118491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 489/964] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Oct 20 14:31:38 2022] GET /identity/v3/role_assignments?user.id=fc18a78229064e8d8ea4321c3c23c4ba&scope.project.id=e49ab75eb6fe499a901d76ddc3ff8027&role.id=d8edc38e326243f28c9b81911ac5ed8f => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:41.282394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e735c7-90b9-43c4-ae8b-f8e52c2b4cf8 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:41.282949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e735c7-90b9-43c4-ae8b-f8e52c2b4cf8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:41.283046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e735c7-90b9-43c4-ae8b-f8e52c2b4cf8 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:41.284423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 476/965] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:31:41.289556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c090939-4c07-4166-ad03-b943531452d8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:41.289973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c090939-4c07-4166-ad03-b943531452d8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:41.290104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c090939-4c07-4166-ad03-b943531452d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:41.308392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-2c090939-4c07-4166-ad03-b943531452d8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:31:41.309559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2c090939-4c07-4166-ad03-b943531452d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:31:41.373766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 490/966] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:31:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:31:41.642865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e735c7-90b9-43c4-ae8b-f8e52c2b4cf8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:41.672719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:41.741092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e735c7-90b9-43c4-ae8b-f8e52c2b4cf8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:41.773280 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1eec6-881b-421b-9aca-5847a03e459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:41.773890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1eec6-881b-421b-9aca-5847a03e459d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:41.774166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1eec6-881b-421b-9aca-5847a03e459d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:41.774431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1eec6-881b-421b-9aca-5847a03e459d None admin] PATH_INFO: `/v3/services/placement` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:41.774993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1eec6-881b-421b-9aca-5847a03e459d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:41.776379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1eec6-881b-421b-9aca-5847a03e459d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-69c1eec6-881b-421b-9aca-5847a03e459d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:41.784747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:41.787575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:41.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 477/967] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Oct 20 14:31:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:41.792534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c090939-4c07-4166-ad03-b943531452d8 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:41.877659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:41.887123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c090939-4c07-4166-ad03-b943531452d8 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:41.918878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49b72f-7428-41c8-9855-9fe486ccf293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:41.919401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b72f-7428-41c8-9855-9fe486ccf293 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:41.919675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b72f-7428-41c8-9855-9fe486ccf293 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:41.919944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b72f-7428-41c8-9855-9fe486ccf293 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:41.920584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49b72f-7428-41c8-9855-9fe486ccf293 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:41.920884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49b72f-7428-41c8-9855-9fe486ccf293 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:41.922229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49b72f-7428-41c8-9855-9fe486ccf293 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:41.927417 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 491/968] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Oct 20 14:31:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:41.933064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1eec6-881b-421b-9aca-5847a03e459d None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:42.039508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:42.046654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1eec6-881b-421b-9aca-5847a03e459d None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:42.080740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31002724-64ab-465f-bcb5-61577468be0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:42.081343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31002724-64ab-465f-bcb5-61577468be0d None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:42.081647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31002724-64ab-465f-bcb5-61577468be0d None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:42.081908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31002724-64ab-465f-bcb5-61577468be0d None admin] PATH_INFO: `/v3/services` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:42.082525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31002724-64ab-465f-bcb5-61577468be0d None admin] RBAC: Adding query filter params (type=placement) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:42.082826 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31002724-64ab-465f-bcb5-61577468be0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:42.084228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31002724-64ab-465f-bcb5-61577468be0d None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:42.088876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 478/969] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Oct 20 14:31:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:44.202264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9af49-ede5-4a41-87bd-ee09cf76c21d None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:44.202789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9af49-ede5-4a41-87bd-ee09cf76c21d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:44.202789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9af49-ede5-4a41-87bd-ee09cf76c21d None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:44.204130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 492/970] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:31:44.209924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a30dd-c6fb-4b9b-97e9-0199500e72e8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:44.210305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a30dd-c6fb-4b9b-97e9-0199500e72e8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:44.210479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a30dd-c6fb-4b9b-97e9-0199500e72e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:44.228291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-555a30dd-c6fb-4b9b-97e9-0199500e72e8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:31:44.229504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-555a30dd-c6fb-4b9b-97e9-0199500e72e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:31:44.294610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 479/971] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:31:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:31:44.573568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9af49-ede5-4a41-87bd-ee09cf76c21d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:44.672119 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9af49-ede5-4a41-87bd-ee09cf76c21d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:44.705471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd876d0-06e3-40d3-99ca-3c6f1a6c2c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:44.706056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd876d0-06e3-40d3-99ca-3c6f1a6c2c76 None admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:44.706334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd876d0-06e3-40d3-99ca-3c6f1a6c2c76 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:44.706732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd876d0-06e3-40d3-99ca-3c6f1a6c2c76 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:44.707744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd876d0-06e3-40d3-99ca-3c6f1a6c2c76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:44.709124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd876d0-06e3-40d3-99ca-3c6f1a6c2c76 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:44.723384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 493/972] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Oct 20 14:31:44 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:31:46.884281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb24eb4-0869-4667-867f-9176312b0823 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:46.884890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb24eb4-0869-4667-867f-9176312b0823 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:46.884890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb24eb4-0869-4667-867f-9176312b0823 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:46.886195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 480/973] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:31:46.891393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fc52b-58ef-498e-8906-de3b22184c9c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:46.891786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fc52b-58ef-498e-8906-de3b22184c9c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:46.891962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fc52b-58ef-498e-8906-de3b22184c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:46.909510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-3b9fc52b-58ef-498e-8906-de3b22184c9c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:31:46.910680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3b9fc52b-58ef-498e-8906-de3b22184c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:31:46.919509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:46.985824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 494/974] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:31:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 14:31:47.270741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb24eb4-0869-4667-867f-9176312b0823 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:47.358996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:47.370587 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb24eb4-0869-4667-867f-9176312b0823 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:47.403482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f0c4e-19e3-4900-a227-9c5b04c42d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:47.404105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f0c4e-19e3-4900-a227-9c5b04c42d07 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:47.404411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f0c4e-19e3-4900-a227-9c5b04c42d07 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:47.404691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f0c4e-19e3-4900-a227-9c5b04c42d07 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:47.405271 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f0c4e-19e3-4900-a227-9c5b04c42d07 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:47.406640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f0c4e-19e3-4900-a227-9c5b04c42d07 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-7a2f0c4e-19e3-4900-a227-9c5b04c42d07 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:47.415176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:47.416416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:47.417509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 481/975] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Oct 20 14:31:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:47.422289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fc52b-58ef-498e-8906-de3b22184c9c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:47.516570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fc52b-58ef-498e-8906-de3b22184c9c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:47.548272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00933-a262-44f3-8943-f5ffcb9dfea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:47.548835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00933-a262-44f3-8943-f5ffcb9dfea7 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:47.549116 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00933-a262-44f3-8943-f5ffcb9dfea7 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:47.549392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00933-a262-44f3-8943-f5ffcb9dfea7 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:47.550010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00933-a262-44f3-8943-f5ffcb9dfea7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:47.550838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00933-a262-44f3-8943-f5ffcb9dfea7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:47.552195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00933-a262-44f3-8943-f5ffcb9dfea7 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:47.557132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 495/976] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Oct 20 14:31:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:47.562864 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f0c4e-19e3-4900-a227-9c5b04c42d07 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:47.660314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f0c4e-19e3-4900-a227-9c5b04c42d07 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:47.693433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef59b-9d6a-4212-af92-7fae94118be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:47.694021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef59b-9d6a-4212-af92-7fae94118be0 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:47.694298 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef59b-9d6a-4212-af92-7fae94118be0 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:47.694571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef59b-9d6a-4212-af92-7fae94118be0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:47.695259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef59b-9d6a-4212-af92-7fae94118be0 None admin] RBAC: Adding query filter params (interface=public, service_id=abdfb1a7587c4eefa1db3b2f1926df95, region_id=RegionOne) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:47.695568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef59b-9d6a-4212-af92-7fae94118be0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:47.697323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef59b-9d6a-4212-af92-7fae94118be0 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:47.702337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 482/977] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Oct 20 14:31:47 2022] GET /identity/v3/endpoints?service_id=abdfb1a7587c4eefa1db3b2f1926df95&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:47.708025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00933-a262-44f3-8943-f5ffcb9dfea7 None admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:47.711674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:47.803577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00933-a262-44f3-8943-f5ffcb9dfea7 None admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:47.835264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d10f6c-6a2d-416d-9157-ef5dfbd14912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:47.835847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d10f6c-6a2d-416d-9157-ef5dfbd14912 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:47.836127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d10f6c-6a2d-416d-9157-ef5dfbd14912 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:47.836428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d10f6c-6a2d-416d-9157-ef5dfbd14912 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:47.837022 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d10f6c-6a2d-416d-9157-ef5dfbd14912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:47.838378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d10f6c-6a2d-416d-9157-ef5dfbd14912 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:47.844902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 496/978] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Oct 20 14:31:47 2022] GET /identity/v3/services => generated 2191 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:31:50.030899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec374a38-b65a-43db-b11d-5249c7c859bd None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:50.031472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec374a38-b65a-43db-b11d-5249c7c859bd None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:50.031583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec374a38-b65a-43db-b11d-5249c7c859bd None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:50.033296 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 483/979] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:31:50.039921 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db8891-904c-42af-829b-5463b7ae2760 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:50.040371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db8891-904c-42af-829b-5463b7ae2760 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:50.040522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db8891-904c-42af-829b-5463b7ae2760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:50.059686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-62db8891-904c-42af-829b-5463b7ae2760 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:31:50.060912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-62db8891-904c-42af-829b-5463b7ae2760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:31:50.119872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:50.125444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 497/980] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:31:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 14:31:50.394684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec374a38-b65a-43db-b11d-5249c7c859bd None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:50.414301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:50.494681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec374a38-b65a-43db-b11d-5249c7c859bd None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:50.527162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ff954-733d-42f7-bb3d-91631e5818a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:50.527769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ff954-733d-42f7-bb3d-91631e5818a9 None admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:50.528045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ff954-733d-42f7-bb3d-91631e5818a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:50.528315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ff954-733d-42f7-bb3d-91631e5818a9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:50.528910 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ff954-733d-42f7-bb3d-91631e5818a9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:50.530252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ff954-733d-42f7-bb3d-91631e5818a9 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application [None req-812ff954-733d-42f7-bb3d-91631e5818a9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 14:31:50.538978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 14:31:50.540126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application with Lock( Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self._enter() Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application return self.creator() Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application created_value = creator( Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: ERROR keystone.server.flask.application Oct 20 14:31:50.541014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 484/981] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Oct 20 14:31:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 14:31:50.546015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db8891-904c-42af-829b-5463b7ae2760 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:50.645016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db8891-904c-42af-829b-5463b7ae2760 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:50.677146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd5f03-34d7-47c4-9445-0f170d51ac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:50.677741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd5f03-34d7-47c4-9445-0f170d51ac44 None admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:50.678018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd5f03-34d7-47c4-9445-0f170d51ac44 None admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:50.678290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd5f03-34d7-47c4-9445-0f170d51ac44 None admin] PATH_INFO: `/v3/services` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:50.678908 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd5f03-34d7-47c4-9445-0f170d51ac44 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:31:50.679209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd5f03-34d7-47c4-9445-0f170d51ac44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:50.680601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd5f03-34d7-47c4-9445-0f170d51ac44 None admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:50.685483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 498/982] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Oct 20 14:31:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:31:50.691819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ff954-733d-42f7-bb3d-91631e5818a9 None admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:31:50.769594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:31:50.787411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ff954-733d-42f7-bb3d-91631e5818a9 None admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:31:50.819323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ba0b2-c0f9-4d4d-b91b-80dc811042e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:31:50.819900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ba0b2-c0f9-4d4d-b91b-80dc811042e6 None admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:31:50.820177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ba0b2-c0f9-4d4d-b91b-80dc811042e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:31:50.820479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ba0b2-c0f9-4d4d-b91b-80dc811042e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:31:50.821135 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ba0b2-c0f9-4d4d-b91b-80dc811042e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:31:50.822488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ba0b2-c0f9-4d4d-b91b-80dc811042e6 None admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:31:50.841155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 485/983] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:31:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:32:00.683034 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=69 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:32:00.837283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:32:54.726847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeffb0-cb21-4df3-ab2c-be603b506294 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:32:54.729080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeffb0-cb21-4df3-ab2c-be603b506294 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:32:54.729648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeffb0-cb21-4df3-ab2c-be603b506294 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:32:54.731955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 499/984] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:32:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:32:54.737937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8680ac0-5e36-4223-9429-80ce0d2e10cf None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:32:54.738423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8680ac0-5e36-4223-9429-80ce0d2e10cf None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:32:54.738529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8680ac0-5e36-4223-9429-80ce0d2e10cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:32:54.839449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-e8680ac0-5e36-4223-9429-80ce0d2e10cf None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:32:54.841035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e8680ac0-5e36-4223-9429-80ce0d2e10cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:32:54.988130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 486/985] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:32:55.270552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0868e7-348f-485c-888e-a12bfec10e7e None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:32:55.271032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0868e7-348f-485c-888e-a12bfec10e7e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:32:55.271385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0868e7-348f-485c-888e-a12bfec10e7e None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:32:55.272869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 500/986] 10.222.0.13 () {54 vars in 964 bytes} [Thu Oct 20 14:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:32:55.279079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d02d2-17bb-42da-85d2-3d539f6bd46f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:32:55.279363 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d02d2-17bb-42da-85d2-3d539f6bd46f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:32:55.279639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d02d2-17bb-42da-85d2-3d539f6bd46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:32:55.369078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-9e9d02d2-17bb-42da-85d2-3d539f6bd46f None None] MFA Rules not processed for user `ae4b023dbf114bf98d0e38ea7fec94fa`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:32:55.370418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9e9d02d2-17bb-42da-85d2-3d539f6bd46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:32:55.392030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:32:55.450746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 487/987] 10.222.0.13 () {58 vars in 1061 bytes} [Thu Oct 20 14:32:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:32:55.459904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0868e7-348f-485c-888e-a12bfec10e7e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:32:55.461788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8c0868e7-348f-485c-888e-a12bfec10e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:32:55.576966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0868e7-348f-485c-888e-a12bfec10e7e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:32:55.659007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456df6c-125b-4cef-8124-60a7dfb443ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:32:55.659628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456df6c-125b-4cef-8124-60a7dfb443ce service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:32:55.659910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456df6c-125b-4cef-8124-60a7dfb443ce service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:32:55.660310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456df6c-125b-4cef-8124-60a7dfb443ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:32:55.661948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5456df6c-125b-4cef-8124-60a7dfb443ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:32:55.722615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:32:55.782771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456df6c-125b-4cef-8124-60a7dfb443ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:32:55.784229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456df6c-125b-4cef-8124-60a7dfb443ce service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:32:55.805565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 501/988] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:32:58.272216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f157-5c4c-4052-9300-552cc251d3e0 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:32:58.272775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f157-5c4c-4052-9300-552cc251d3e0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:32:58.272775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f157-5c4c-4052-9300-552cc251d3e0 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:32:58.274120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 488/989] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:32:58.279202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b2429-17c9-4633-9489-986908c8ebc8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:32:58.279594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b2429-17c9-4633-9489-986908c8ebc8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:32:58.279757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b2429-17c9-4633-9489-986908c8ebc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:32:58.361692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-cd6b2429-17c9-4633-9489-986908c8ebc8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:32:58.363156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-cd6b2429-17c9-4633-9489-986908c8ebc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:32:58.456019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 502/990] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:32:58.737931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269f157-5c4c-4052-9300-552cc251d3e0 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:32:58.739467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3269f157-5c4c-4052-9300-552cc251d3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:32:58.852335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269f157-5c4c-4052-9300-552cc251d3e0 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:32:58.924072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:32:58.926681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2716bbd-59d3-4bdb-8f09-05e1239f034e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:32:58.927490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2716bbd-59d3-4bdb-8f09-05e1239f034e service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:32:58.927931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2716bbd-59d3-4bdb-8f09-05e1239f034e service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:32:58.928401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2716bbd-59d3-4bdb-8f09-05e1239f034e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:32:58.929994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b2716bbd-59d3-4bdb-8f09-05e1239f034e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:32:59.040042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2716bbd-59d3-4bdb-8f09-05e1239f034e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:32:59.041519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2716bbd-59d3-4bdb-8f09-05e1239f034e service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:32:59.059762 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 489/991] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:32:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:01.604703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b472013-b2cf-4ec0-9d8a-bc647259b08b None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:01.605239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b472013-b2cf-4ec0-9d8a-bc647259b08b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:01.605239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b472013-b2cf-4ec0-9d8a-bc647259b08b None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:01.606497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 503/992] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:01.612770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bc38b-9c53-4de4-a4e8-9cd98205579c None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:01.613176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bc38b-9c53-4de4-a4e8-9cd98205579c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:01.613360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bc38b-9c53-4de4-a4e8-9cd98205579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:01.677563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ff8bc38b-9c53-4de4-a4e8-9cd98205579c None None] MFA Rules not processed for user `29b4871936674640986d3796e5af8a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:01.678753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ff8bc38b-9c53-4de4-a4e8-9cd98205579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:01.728532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:01.773440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 490/993] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:02.081682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609770bf-7157-4dc3-b7d8-daea206dbea9 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:02.081966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609770bf-7157-4dc3-b7d8-daea206dbea9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:02.082245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609770bf-7157-4dc3-b7d8-daea206dbea9 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:02.083566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 504/994] 10.222.0.13 () {54 vars in 964 bytes} [Thu Oct 20 14:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:02.089230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e80b4f-e584-4cc1-9979-2272e2fa5450 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:02.089513 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e80b4f-e584-4cc1-9979-2272e2fa5450 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:02.089787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e80b4f-e584-4cc1-9979-2272e2fa5450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:02.163960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-11e80b4f-e584-4cc1-9979-2272e2fa5450 None None] MFA Rules not processed for user `ae4b023dbf114bf98d0e38ea7fec94fa`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:02.165125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-11e80b4f-e584-4cc1-9979-2272e2fa5450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:02.240862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 491/995] 10.222.0.13 () {58 vars in 1061 bytes} [Thu Oct 20 14:33:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:02.249817 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609770bf-7157-4dc3-b7d8-daea206dbea9 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:02.251372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-609770bf-7157-4dc3-b7d8-daea206dbea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:02.260992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:02.352455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609770bf-7157-4dc3-b7d8-daea206dbea9 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:02.425442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4d6fe-b5c5-4b37-abc9-170927367fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:02.425973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d6fe-b5c5-4b37-abc9-170927367fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:02.426252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d6fe-b5c5-4b37-abc9-170927367fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:02.426526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d6fe-b5c5-4b37-abc9-170927367fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:02.428107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-98a4d6fe-b5c5-4b37-abc9-170927367fd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:02.493416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:02.540131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4d6fe-b5c5-4b37-abc9-170927367fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:02.541599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4d6fe-b5c5-4b37-abc9-170927367fd9 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:02.559482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 505/996] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:05.606447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360638e7-6286-45cc-b7e9-74d37f8f774b None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:05.606950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360638e7-6286-45cc-b7e9-74d37f8f774b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:05.606950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360638e7-6286-45cc-b7e9-74d37f8f774b None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:05.608247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 492/997] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:05.614097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872c3f0-c7f6-4da3-a96a-a6b0bb5bf553 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:05.614535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872c3f0-c7f6-4da3-a96a-a6b0bb5bf553 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:05.614652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872c3f0-c7f6-4da3-a96a-a6b0bb5bf553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:05.679381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-5872c3f0-c7f6-4da3-a96a-a6b0bb5bf553 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:05.680610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5872c3f0-c7f6-4da3-a96a-a6b0bb5bf553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:05.774667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 506/998] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:33:06.064897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360638e7-6286-45cc-b7e9-74d37f8f774b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:06.066451 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-360638e7-6286-45cc-b7e9-74d37f8f774b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:06.115631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:06.167336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360638e7-6286-45cc-b7e9-74d37f8f774b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:06.237027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63a322-ff8e-4e29-a995-9296e64274d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:06.237886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63a322-ff8e-4e29-a995-9296e64274d4 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:06.238161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63a322-ff8e-4e29-a995-9296e64274d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:06.238430 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63a322-ff8e-4e29-a995-9296e64274d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:06.239988 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-aa63a322-ff8e-4e29-a995-9296e64274d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:06.330746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:06.344531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63a322-ff8e-4e29-a995-9296e64274d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:06.345974 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63a322-ff8e-4e29-a995-9296e64274d4 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:06.364453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 493/999] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:09.158789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe781a-dd47-4d05-8af7-b173cea26ba7 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:09.159348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe781a-dd47-4d05-8af7-b173cea26ba7 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:09.159348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe781a-dd47-4d05-8af7-b173cea26ba7 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:09.160759 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 507/1000] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:09.165977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e59b4b-df59-400c-8ebe-f69132c2a00f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:09.166426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e59b4b-df59-400c-8ebe-f69132c2a00f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:09.166519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e59b4b-df59-400c-8ebe-f69132c2a00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:09.238996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-37e59b4b-df59-400c-8ebe-f69132c2a00f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:09.240225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-37e59b4b-df59-400c-8ebe-f69132c2a00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:09.346585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 494/1001] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:09.633165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe781a-dd47-4d05-8af7-b173cea26ba7 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:09.634697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-29fe781a-dd47-4d05-8af7-b173cea26ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:09.654856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:09.738651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe781a-dd47-4d05-8af7-b173cea26ba7 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:09.810620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1d17e-7af6-4525-b212-107de6e8212e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:09.811217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1d17e-7af6-4525-b212-107de6e8212e service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:09.811495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1d17e-7af6-4525-b212-107de6e8212e service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:09.811766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1d17e-7af6-4525-b212-107de6e8212e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:09.813909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-84f1d17e-7af6-4525-b212-107de6e8212e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:09.886018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:09.925792 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1d17e-7af6-4525-b212-107de6e8212e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:09.927223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1d17e-7af6-4525-b212-107de6e8212e service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:09.945848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 508/1002] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:12.158498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585444dd-ae26-45b2-9301-f7bdc11da069 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:12.159016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585444dd-ae26-45b2-9301-f7bdc11da069 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:12.159016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585444dd-ae26-45b2-9301-f7bdc11da069 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:12.160298 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 495/1003] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:33:12.165978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d201360-eb51-48e7-9585-431a42fb7bf9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:12.166409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d201360-eb51-48e7-9585-431a42fb7bf9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:12.166546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d201360-eb51-48e7-9585-431a42fb7bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:12.231939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4d201360-eb51-48e7-9585-431a42fb7bf9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:12.233241 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4d201360-eb51-48e7-9585-431a42fb7bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:12.338655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 509/1004] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:12.626470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585444dd-ae26-45b2-9301-f7bdc11da069 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:12.628008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-585444dd-ae26-45b2-9301-f7bdc11da069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:12.688849 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:12.731322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585444dd-ae26-45b2-9301-f7bdc11da069 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:12.804577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed20b9-5530-4202-9d25-f5cd5d857e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:12.805184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed20b9-5530-4202-9d25-f5cd5d857e0a service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:12.805464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed20b9-5530-4202-9d25-f5cd5d857e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:12.805735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed20b9-5530-4202-9d25-f5cd5d857e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:12.807315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e1ed20b9-5530-4202-9d25-f5cd5d857e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:12.915735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:12.918133 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed20b9-5530-4202-9d25-f5cd5d857e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:12.920104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed20b9-5530-4202-9d25-f5cd5d857e0a service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:12.939110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 496/1005] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:15.133491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f182310-a846-4115-8213-13a214877b3e None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:15.134021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f182310-a846-4115-8213-13a214877b3e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:15.134110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f182310-a846-4115-8213-13a214877b3e None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:15.135382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 510/1006] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:15.140548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21251385-ac6a-4dc0-ad5d-525ede0e5f89 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:15.140979 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21251385-ac6a-4dc0-ad5d-525ede0e5f89 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:15.141262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21251385-ac6a-4dc0-ad5d-525ede0e5f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:15.207304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-21251385-ac6a-4dc0-ad5d-525ede0e5f89 None None] MFA Rules not processed for user `29b4871936674640986d3796e5af8a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:15.208535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-21251385-ac6a-4dc0-ad5d-525ede0e5f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:15.305305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 497/1007] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:15.584708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f182310-a846-4115-8213-13a214877b3e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:15.586269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0f182310-a846-4115-8213-13a214877b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:15.620034 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:15.696721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f182310-a846-4115-8213-13a214877b3e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:15.772083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bc85c-4556-46e2-9323-6bf294c04b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:15.772714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bc85c-4556-46e2-9323-6bf294c04b7f service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:15.772991 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bc85c-4556-46e2-9323-6bf294c04b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:15.773258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bc85c-4556-46e2-9323-6bf294c04b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:15.774815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-700bc85c-4556-46e2-9323-6bf294c04b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:15.851101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:15.883516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bc85c-4556-46e2-9323-6bf294c04b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:15.884989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bc85c-4556-46e2-9323-6bf294c04b7f service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:15.903074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 511/1008] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:18.336290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996d82-d0ea-4888-96d1-4d52a9e2f39d None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:18.336911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996d82-d0ea-4888-96d1-4d52a9e2f39d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:18.337012 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996d82-d0ea-4888-96d1-4d52a9e2f39d None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:18.338321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 498/1009] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:18.344365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958d0e1-6c5f-4298-8019-16c9caf77886 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:18.344819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958d0e1-6c5f-4298-8019-16c9caf77886 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:18.344917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958d0e1-6c5f-4298-8019-16c9caf77886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:18.408714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-b958d0e1-6c5f-4298-8019-16c9caf77886 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:18.409905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b958d0e1-6c5f-4298-8019-16c9caf77886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:18.502845 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 512/1010] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:18.784462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0996d82-d0ea-4888-96d1-4d52a9e2f39d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:18.785999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f0996d82-d0ea-4888-96d1-4d52a9e2f39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:18.855745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:18.889424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0996d82-d0ea-4888-96d1-4d52a9e2f39d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:18.962416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc552ee0-3f92-4198-87ef-0615534e5c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:18.963014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc552ee0-3f92-4198-87ef-0615534e5c18 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:18.963289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc552ee0-3f92-4198-87ef-0615534e5c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:18.963565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc552ee0-3f92-4198-87ef-0615534e5c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:18.965171 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fc552ee0-3f92-4198-87ef-0615534e5c18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:19.071493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc552ee0-3f92-4198-87ef-0615534e5c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:19.072896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc552ee0-3f92-4198-87ef-0615534e5c18 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:19.083257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:19.091602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 499/1011] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:21.255918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24ae7-9690-42ff-a20c-65f10601df25 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:21.256484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24ae7-9690-42ff-a20c-65f10601df25 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:21.256484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24ae7-9690-42ff-a20c-65f10601df25 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:21.257831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 513/1012] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:21.263234 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dc7d6-fc44-4c41-8942-6abee0691469 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:21.263657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dc7d6-fc44-4c41-8942-6abee0691469 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:21.263782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dc7d6-fc44-4c41-8942-6abee0691469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:21.327279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-453dc7d6-fc44-4c41-8942-6abee0691469 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:21.328763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-453dc7d6-fc44-4c41-8942-6abee0691469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:21.423308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 500/1013] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:21.709406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24ae7-9690-42ff-a20c-65f10601df25 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:21.710951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fac24ae7-9690-42ff-a20c-65f10601df25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:21.751306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:21.813158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24ae7-9690-42ff-a20c-65f10601df25 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:21.884010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6d4a2-81fd-487e-8fca-245194bfa15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:21.884623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d4a2-81fd-487e-8fca-245194bfa15e service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:21.884897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d4a2-81fd-487e-8fca-245194bfa15e service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:21.885163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d4a2-81fd-487e-8fca-245194bfa15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:21.886709 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d3e6d4a2-81fd-487e-8fca-245194bfa15e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:21.969261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:21.995984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d4a2-81fd-487e-8fca-245194bfa15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:21.997473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d4a2-81fd-487e-8fca-245194bfa15e service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:22.016275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 514/1014] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:24.496694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5e6b6-773c-4ddb-a566-2c2161531ea4 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:24.497243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5e6b6-773c-4ddb-a566-2c2161531ea4 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:24.497243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5e6b6-773c-4ddb-a566-2c2161531ea4 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:24.498631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 501/1015] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:24.504132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7ad48-ca2d-4b73-932f-58ade26d7265 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:24.504581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7ad48-ca2d-4b73-932f-58ade26d7265 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:24.504716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7ad48-ca2d-4b73-932f-58ade26d7265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:24.568581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-abe7ad48-ca2d-4b73-932f-58ade26d7265 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:24.569776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-abe7ad48-ca2d-4b73-932f-58ade26d7265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:24.663029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 515/1016] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:24.945346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5e6b6-773c-4ddb-a566-2c2161531ea4 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:24.946880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-29d5e6b6-773c-4ddb-a566-2c2161531ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:25.024482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:25.052810 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5e6b6-773c-4ddb-a566-2c2161531ea4 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:25.126113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157506f-fa16-4572-bd67-ebdb590c79de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:25.126705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157506f-fa16-4572-bd67-ebdb590c79de service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:25.126980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157506f-fa16-4572-bd67-ebdb590c79de service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:25.127247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157506f-fa16-4572-bd67-ebdb590c79de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:25.128843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4157506f-fa16-4572-bd67-ebdb590c79de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:25.233279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157506f-fa16-4572-bd67-ebdb590c79de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:25.235297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157506f-fa16-4572-bd67-ebdb590c79de service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:25.252803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 502/1017] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:29.568996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734f7cb-e881-4f8c-8a28-3e23e6af861f None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:29.569513 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734f7cb-e881-4f8c-8a28-3e23e6af861f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:29.569513 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734f7cb-e881-4f8c-8a28-3e23e6af861f None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:29.570787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 516/1018] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:29.575988 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b9137-731e-4c2b-a620-8c1d98374d41 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:29.576413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b9137-731e-4c2b-a620-8c1d98374d41 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:29.576578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b9137-731e-4c2b-a620-8c1d98374d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:29.582166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:29.645666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-7b6b9137-731e-4c2b-a620-8c1d98374d41 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:29.646870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7b6b9137-731e-4c2b-a620-8c1d98374d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:29.744586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 503/1019] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:30.029299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734f7cb-e881-4f8c-8a28-3e23e6af861f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:30.031429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4734f7cb-e881-4f8c-8a28-3e23e6af861f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:30.080079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:30.141188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734f7cb-e881-4f8c-8a28-3e23e6af861f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:30.214490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb2416-1438-4131-bcc8-bc6e981461d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:30.215103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb2416-1438-4131-bcc8-bc6e981461d6 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:30.215382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb2416-1438-4131-bcc8-bc6e981461d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:30.215668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb2416-1438-4131-bcc8-bc6e981461d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:30.217332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4acb2416-1438-4131-bcc8-bc6e981461d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:30.310726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:30.328005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb2416-1438-4131-bcc8-bc6e981461d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:30.329506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb2416-1438-4131-bcc8-bc6e981461d6 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:30.354399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 517/1020] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:33.176865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f62f73-6e3b-4e3d-b64e-839bc7cb69e8 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:33.177399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f62f73-6e3b-4e3d-b64e-839bc7cb69e8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:33.177399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f62f73-6e3b-4e3d-b64e-839bc7cb69e8 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:33.178771 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 504/1021] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:33.184106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5f04e-63c0-4a12-a7a3-eb5105cc5b4f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:33.184547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5f04e-63c0-4a12-a7a3-eb5105cc5b4f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:33.184695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5f04e-63c0-4a12-a7a3-eb5105cc5b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:33.250381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c1a5f04e-63c0-4a12-a7a3-eb5105cc5b4f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:33.251567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c1a5f04e-63c0-4a12-a7a3-eb5105cc5b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:33.355880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 518/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:33.641257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f62f73-6e3b-4e3d-b64e-839bc7cb69e8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:33.642809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-77f62f73-6e3b-4e3d-b64e-839bc7cb69e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:33.730456 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:33.747935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f62f73-6e3b-4e3d-b64e-839bc7cb69e8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:33.820724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2a664-cf55-42f1-9e80-b77e426618c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:33.821233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2a664-cf55-42f1-9e80-b77e426618c8 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:33.821508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2a664-cf55-42f1-9e80-b77e426618c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:33.821778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2a664-cf55-42f1-9e80-b77e426618c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:33.823342 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1ab2a664-cf55-42f1-9e80-b77e426618c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:33.932877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2a664-cf55-42f1-9e80-b77e426618c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:33.934323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2a664-cf55-42f1-9e80-b77e426618c8 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:33.953668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 505/1023] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:38.107864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cea5dd-7269-464a-8939-f3b555288a5a None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:38.108848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cea5dd-7269-464a-8939-f3b555288a5a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:38.109098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cea5dd-7269-464a-8939-f3b555288a5a None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:38.110306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 519/1024] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:38.116227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b0d30-8d50-4530-8fc8-e771f3eeb42a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:38.116912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b0d30-8d50-4530-8fc8-e771f3eeb42a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:38.116912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b0d30-8d50-4530-8fc8-e771f3eeb42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:38.136105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:38.189675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d00b0d30-8d50-4530-8fc8-e771f3eeb42a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:38.190973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d00b0d30-8d50-4530-8fc8-e771f3eeb42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:38.309453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 506/1025] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:38.589472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cea5dd-7269-464a-8939-f3b555288a5a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:38.590941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-85cea5dd-7269-464a-8939-f3b555288a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:38.648435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:38.693533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cea5dd-7269-464a-8939-f3b555288a5a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:38.765719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c698df-f6b3-4eee-8756-138dc2181948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:38.766322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c698df-f6b3-4eee-8756-138dc2181948 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:38.766606 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c698df-f6b3-4eee-8756-138dc2181948 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:38.767125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c698df-f6b3-4eee-8756-138dc2181948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:38.768722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-09c698df-f6b3-4eee-8756-138dc2181948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:38.873628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:38.876868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c698df-f6b3-4eee-8756-138dc2181948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:38.878623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c698df-f6b3-4eee-8756-138dc2181948 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:38.896703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 520/1026] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:43.734955 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69fb1b-1e05-4203-b8f4-b5845c2ca3cc None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:43.735393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69fb1b-1e05-4203-b8f4-b5845c2ca3cc None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:43.738169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69fb1b-1e05-4203-b8f4-b5845c2ca3cc None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:43.738169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 507/1027] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:43.743134 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a45223-001e-47d9-9a70-5e331a5c6493 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:43.744538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a45223-001e-47d9-9a70-5e331a5c6493 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:43.744886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a45223-001e-47d9-9a70-5e331a5c6493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:43.854064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d8a45223-001e-47d9-9a70-5e331a5c6493 None None] MFA Rules not processed for user `fc18a78229064e8d8ea4321c3c23c4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:43.855085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d8a45223-001e-47d9-9a70-5e331a5c6493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:44.019713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 521/1028] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:33:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:48.308085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=59 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:48.366467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceea309-bd97-4cdd-96f5-a2807d2afdbb None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:48.366756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceea309-bd97-4cdd-96f5-a2807d2afdbb None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:48.367024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceea309-bd97-4cdd-96f5-a2807d2afdbb None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:48.368381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 508/1029] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:48.373713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9715-e313-4bcd-9dc2-3dde720cbf44 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:48.374190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9715-e313-4bcd-9dc2-3dde720cbf44 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:48.374481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9715-e313-4bcd-9dc2-3dde720cbf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:48.433399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:48.460899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-094e9715-e313-4bcd-9dc2-3dde720cbf44 None None] MFA Rules not processed for user `fc18a78229064e8d8ea4321c3c23c4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:48.462136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-094e9715-e313-4bcd-9dc2-3dde720cbf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:48.584497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 522/1030] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:33:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:33:49.891865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e199422-7321-4f2a-aef6-46d4c2d6d805 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:49.892471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e199422-7321-4f2a-aef6-46d4c2d6d805 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:49.892471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e199422-7321-4f2a-aef6-46d4c2d6d805 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:49.893790 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 509/1031] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:49.898811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611b19-0f5e-4218-9a33-7175b627b96b None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:49.899230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611b19-0f5e-4218-9a33-7175b627b96b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:49.899539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611b19-0f5e-4218-9a33-7175b627b96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:49.979293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-74611b19-0f5e-4218-9a33-7175b627b96b None None] MFA Rules not processed for user `fc18a78229064e8d8ea4321c3c23c4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:49.980618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-74611b19-0f5e-4218-9a33-7175b627b96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:50.027126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:50.097604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 523/1032] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:33:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:33:53.541699 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb799f-3490-4abd-9a28-304dc5ec2469 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:53.542235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb799f-3490-4abd-9a28-304dc5ec2469 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:53.542235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb799f-3490-4abd-9a28-304dc5ec2469 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:53.543583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 510/1033] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:53.549442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7cd89-0dca-4f19-a377-104ca4e0b641 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:53.549895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7cd89-0dca-4f19-a377-104ca4e0b641 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:53.550332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7cd89-0dca-4f19-a377-104ca4e0b641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:53.636015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-35c7cd89-0dca-4f19-a377-104ca4e0b641 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:53.637577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-35c7cd89-0dca-4f19-a377-104ca4e0b641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:53.742636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 524/1034] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:54.053775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fba3a9-0b32-4485-a71d-b9f681bdfd3d None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:54.053948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fba3a9-0b32-4485-a71d-b9f681bdfd3d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:54.054219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fba3a9-0b32-4485-a71d-b9f681bdfd3d None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:54.055571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 511/1035] 10.222.0.13 () {54 vars in 961 bytes} [Thu Oct 20 14:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:54.065474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f62f7-e2ce-43db-a665-6ccc7ec7cdd2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:54.065474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f62f7-e2ce-43db-a665-6ccc7ec7cdd2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:54.065668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f62f7-e2ce-43db-a665-6ccc7ec7cdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:54.087319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:54.161525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-8c1f62f7-e2ce-43db-a665-6ccc7ec7cdd2 None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:54.162753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8c1f62f7-e2ce-43db-a665-6ccc7ec7cdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:54.299148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 525/1036] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Oct 20 14:33:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:54.307403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fba3a9-0b32-4485-a71d-b9f681bdfd3d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:54.308933 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-20fba3a9-0b32-4485-a71d-b9f681bdfd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:54.471341 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fba3a9-0b32-4485-a71d-b9f681bdfd3d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:54.551980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:54.598893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc78090-2681-4385-8bdb-63d0dda36b41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:54.599649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78090-2681-4385-8bdb-63d0dda36b41 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:54.600027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78090-2681-4385-8bdb-63d0dda36b41 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:54.600434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78090-2681-4385-8bdb-63d0dda36b41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:54.602157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3cc78090-2681-4385-8bdb-63d0dda36b41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:54.697216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc78090-2681-4385-8bdb-63d0dda36b41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:54.698967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc78090-2681-4385-8bdb-63d0dda36b41 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:54.718445 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 512/1037] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:33:57.107511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaa71b-9f77-4a6e-befc-4e08fd889c88 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:57.108204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaa71b-9f77-4a6e-befc-4e08fd889c88 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:57.108204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaa71b-9f77-4a6e-befc-4e08fd889c88 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:57.113751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 526/1038] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:33:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:33:57.115028 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2b722-48e5-4b7e-afbd-1f5ebef8ff5a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:57.115487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2b722-48e5-4b7e-afbd-1f5ebef8ff5a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:57.115487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2b722-48e5-4b7e-afbd-1f5ebef8ff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:57.186174 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-34e2b722-48e5-4b7e-afbd-1f5ebef8ff5a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:33:57.187399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-34e2b722-48e5-4b7e-afbd-1f5ebef8ff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:57.227253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:57.296553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 513/1039] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:33:57.590158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaa71b-9f77-4a6e-befc-4e08fd889c88 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:33:57.592516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5cbaa71b-9f77-4a6e-befc-4e08fd889c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:57.662939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:57.767493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaa71b-9f77-4a6e-befc-4e08fd889c88 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:33:57.906864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a4bcf-871c-415c-885c-2893138b70d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:33:57.907405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a4bcf-871c-415c-885c-2893138b70d5 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:33:57.907682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a4bcf-871c-415c-885c-2893138b70d5 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:33:57.907954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a4bcf-871c-415c-885c-2893138b70d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:33:57.909629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ca3a4bcf-871c-415c-885c-2893138b70d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:33:57.935902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:33:58.007137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a4bcf-871c-415c-885c-2893138b70d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:33:58.008795 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a4bcf-871c-415c-885c-2893138b70d5 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:33:58.027784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 527/1040] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:00.519275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118604ef-fd7c-4d6a-a4ba-b4e849080b6b None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:00.519810 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118604ef-fd7c-4d6a-a4ba-b4e849080b6b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:00.519810 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118604ef-fd7c-4d6a-a4ba-b4e849080b6b None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:00.521340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 514/1041] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:00.526525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60d2c5-0598-474a-8b96-6bb66b7bae82 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:00.526917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60d2c5-0598-474a-8b96-6bb66b7bae82 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:00.527135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60d2c5-0598-474a-8b96-6bb66b7bae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:00.594650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9b60d2c5-0598-474a-8b96-6bb66b7bae82 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:00.595755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9b60d2c5-0598-474a-8b96-6bb66b7bae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:00.697275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:00.708749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 528/1042] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:00.993225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3ada0-479f-4070-a22b-ae8482685292 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:00.993659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3ada0-479f-4070-a22b-ae8482685292 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:00.994032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3ada0-479f-4070-a22b-ae8482685292 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:00.995652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 515/1043] 10.222.0.13 () {54 vars in 961 bytes} [Thu Oct 20 14:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:01.001678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186434d1-3731-483d-974b-d926725a5d0a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:01.001965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186434d1-3731-483d-974b-d926725a5d0a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:01.002240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186434d1-3731-483d-974b-d926725a5d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:01.081693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-186434d1-3731-483d-974b-d926725a5d0a None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:01.082865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-186434d1-3731-483d-974b-d926725a5d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:01.203503 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 529/1044] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Oct 20 14:34:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:01.211567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3ada0-479f-4070-a22b-ae8482685292 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:01.213229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2da3ada0-479f-4070-a22b-ae8482685292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:01.376810 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3ada0-479f-4070-a22b-ae8482685292 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:01.397413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:01.497791 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10208672-f314-403c-ab6a-f9f7b1651b49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:01.498139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10208672-f314-403c-ab6a-f9f7b1651b49 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:01.498388 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10208672-f314-403c-ab6a-f9f7b1651b49 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:01.498658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10208672-f314-403c-ab6a-f9f7b1651b49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:01.500230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-10208672-f314-403c-ab6a-f9f7b1651b49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:01.599304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10208672-f314-403c-ab6a-f9f7b1651b49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:01.601019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10208672-f314-403c-ab6a-f9f7b1651b49 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:01.622951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 516/1045] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:01.701467 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf9524-be66-435f-b6b5-733586596010 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:01.702019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf9524-be66-435f-b6b5-733586596010 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:01.702019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf9524-be66-435f-b6b5-733586596010 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:01.704385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 530/1046] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:01.709498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9081193-7b21-4cc6-b85b-45cc807a20d6 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:01.709812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9081193-7b21-4cc6-b85b-45cc807a20d6 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:01.710055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9081193-7b21-4cc6-b85b-45cc807a20d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:01.748288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:01.794992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-c9081193-7b21-4cc6-b85b-45cc807a20d6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:01.796522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c9081193-7b21-4cc6-b85b-45cc807a20d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:01.938828 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 517/1047] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:02.299458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cabfd-e313-4806-97d5-88e76d7af70d None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:02.299458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cabfd-e313-4806-97d5-88e76d7af70d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:02.299458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cabfd-e313-4806-97d5-88e76d7af70d None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:02.301180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 531/1048] 10.222.0.13 () {54 vars in 963 bytes} [Thu Oct 20 14:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:02.306303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf84ed-70ab-4f4b-bcbb-fcebf96901ab None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:02.306588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf84ed-70ab-4f4b-bcbb-fcebf96901ab None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:02.306859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf84ed-70ab-4f4b-bcbb-fcebf96901ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:02.395771 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-75cf84ed-70ab-4f4b-bcbb-fcebf96901ab None None] MFA Rules not processed for user `70742ba8f3f6461f8f578b323afea7cc`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:02.397117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-75cf84ed-70ab-4f4b-bcbb-fcebf96901ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:02.456590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:02.491838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 518/1049] 10.222.0.13 () {58 vars in 1060 bytes} [Thu Oct 20 14:34:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:34:02.501697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cabfd-e313-4806-97d5-88e76d7af70d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:02.503401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d19cabfd-e313-4806-97d5-88e76d7af70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:02.556008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:02.613190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cabfd-e313-4806-97d5-88e76d7af70d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:02.696240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743ec7f-a1ff-4739-b629-fa1c767f1e51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:02.696960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743ec7f-a1ff-4739-b629-fa1c767f1e51 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:02.697242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743ec7f-a1ff-4739-b629-fa1c767f1e51 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:02.697518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743ec7f-a1ff-4739-b629-fa1c767f1e51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:02.699135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a743ec7f-a1ff-4739-b629-fa1c767f1e51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:02.810661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:02.827468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743ec7f-a1ff-4739-b629-fa1c767f1e51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:02.828964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743ec7f-a1ff-4739-b629-fa1c767f1e51 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:02.861955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 532/1050] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:34:03.937025 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba14826-1173-4ce5-8184-cfa8c1d2622d None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:03.937393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba14826-1173-4ce5-8184-cfa8c1d2622d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:03.937648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba14826-1173-4ce5-8184-cfa8c1d2622d None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:03.938890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 519/1051] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:03.944192 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa88dd7-26be-4a65-91ce-aa82cee60c24 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:03.944631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa88dd7-26be-4a65-91ce-aa82cee60c24 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:03.944916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa88dd7-26be-4a65-91ce-aa82cee60c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:04.018384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9fa88dd7-26be-4a65-91ce-aa82cee60c24 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:04.019628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9fa88dd7-26be-4a65-91ce-aa82cee60c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:04.126961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 533/1052] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:34:04.422777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba14826-1173-4ce5-8184-cfa8c1d2622d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:04.424304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6ba14826-1173-4ce5-8184-cfa8c1d2622d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:04.579629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba14826-1173-4ce5-8184-cfa8c1d2622d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:04.633095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:04.695574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f5b4-99f9-4047-ac7b-0e4c9bbaea68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:04.696181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f5b4-99f9-4047-ac7b-0e4c9bbaea68 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:04.696492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f5b4-99f9-4047-ac7b-0e4c9bbaea68 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:04.696767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f5b4-99f9-4047-ac7b-0e4c9bbaea68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:04.698349 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-94e3f5b4-99f9-4047-ac7b-0e4c9bbaea68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:04.792316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3f5b4-99f9-4047-ac7b-0e4c9bbaea68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:04.794896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3f5b4-99f9-4047-ac7b-0e4c9bbaea68 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:04.812873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 520/1053] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:07.033512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbda6d-177b-4025-9d07-05ab127cb999 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:07.034068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbda6d-177b-4025-9d07-05ab127cb999 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:07.034327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbda6d-177b-4025-9d07-05ab127cb999 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:07.035919 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 534/1054] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:07.041080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a32cb-0d85-4244-8a1d-42f567baee02 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:07.041517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a32cb-0d85-4244-8a1d-42f567baee02 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:07.041681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a32cb-0d85-4244-8a1d-42f567baee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:07.113347 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-0a9a32cb-0d85-4244-8a1d-42f567baee02 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:07.114547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0a9a32cb-0d85-4244-8a1d-42f567baee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:07.121382 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:07.226189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 521/1055] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:07.527412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbda6d-177b-4025-9d07-05ab127cb999 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:07.529453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3fdbda6d-177b-4025-9d07-05ab127cb999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:07.593433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:07.681369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbda6d-177b-4025-9d07-05ab127cb999 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:07.797347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb838dc7-5f81-456c-9b98-5480f1909ebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:07.797951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb838dc7-5f81-456c-9b98-5480f1909ebf service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:07.798232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb838dc7-5f81-456c-9b98-5480f1909ebf service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:07.798506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb838dc7-5f81-456c-9b98-5480f1909ebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:07.800086 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bb838dc7-5f81-456c-9b98-5480f1909ebf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:07.818940 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:07.895506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb838dc7-5f81-456c-9b98-5480f1909ebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:07.897138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb838dc7-5f81-456c-9b98-5480f1909ebf service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:07.916067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 535/1056] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:10.150290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d735755-003f-4091-9018-80c42b5e801e None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:10.150795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d735755-003f-4091-9018-80c42b5e801e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:10.150888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d735755-003f-4091-9018-80c42b5e801e None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:10.152321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 522/1057] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:10.157456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e45e1-3917-47b7-b469-25b39bcb8808 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:10.157884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e45e1-3917-47b7-b469-25b39bcb8808 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:10.158113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e45e1-3917-47b7-b469-25b39bcb8808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:10.227681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-773e45e1-3917-47b7-b469-25b39bcb8808 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:10.228903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-773e45e1-3917-47b7-b469-25b39bcb8808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:10.328588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:10.359994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 536/1058] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:10.641824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d735755-003f-4091-9018-80c42b5e801e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:10.643394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4d735755-003f-4091-9018-80c42b5e801e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:10.781769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:10.792566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d735755-003f-4091-9018-80c42b5e801e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:10.908446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83184a11-d3e0-4cb4-9a28-91d3e2a6643b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:10.909064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83184a11-d3e0-4cb4-9a28-91d3e2a6643b service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:10.909344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83184a11-d3e0-4cb4-9a28-91d3e2a6643b service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:10.909616 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83184a11-d3e0-4cb4-9a28-91d3e2a6643b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:10.911202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-83184a11-d3e0-4cb4-9a28-91d3e2a6643b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:11.008800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83184a11-d3e0-4cb4-9a28-91d3e2a6643b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:11.010650 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83184a11-d3e0-4cb4-9a28-91d3e2a6643b service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:11.022315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:11.031862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 523/1059] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:13.491526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ace7-56e7-441f-acbc-7c137a24276b None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:13.492057 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ace7-56e7-441f-acbc-7c137a24276b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:13.492238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ace7-56e7-441f-acbc-7c137a24276b None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:13.493984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 537/1060] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:13.500908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c74d71-6e56-4033-9e86-d586f1b899a3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:13.501334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c74d71-6e56-4033-9e86-d586f1b899a3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:13.501455 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c74d71-6e56-4033-9e86-d586f1b899a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:13.569855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-65c74d71-6e56-4033-9e86-d586f1b899a3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:13.571411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-65c74d71-6e56-4033-9e86-d586f1b899a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:13.674251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 524/1061] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:13.959773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ace7-56e7-441f-acbc-7c137a24276b None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:13.961750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f1b4ace7-56e7-441f-acbc-7c137a24276b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:14.128660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ace7-56e7-441f-acbc-7c137a24276b None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:14.203258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:14.259151 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566d80b-7d14-4354-ba99-47f0a8dc39f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:14.259905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566d80b-7d14-4354-ba99-47f0a8dc39f1 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:14.260550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566d80b-7d14-4354-ba99-47f0a8dc39f1 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:14.261186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566d80b-7d14-4354-ba99-47f0a8dc39f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:14.263534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5566d80b-7d14-4354-ba99-47f0a8dc39f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:14.370064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566d80b-7d14-4354-ba99-47f0a8dc39f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:14.371694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566d80b-7d14-4354-ba99-47f0a8dc39f1 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:14.390482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 538/1062] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:16.517211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399f75d-e559-46f4-a630-2d59aa47b83f None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:16.517754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399f75d-e559-46f4-a630-2d59aa47b83f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:16.517860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399f75d-e559-46f4-a630-2d59aa47b83f None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:16.519202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 525/1063] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:16.524735 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374f702-ba2a-47ce-b993-01be64020b6a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:16.525213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374f702-ba2a-47ce-b993-01be64020b6a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:16.525320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374f702-ba2a-47ce-b993-01be64020b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:16.591645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e374f702-ba2a-47ce-b993-01be64020b6a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:16.592863 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e374f702-ba2a-47ce-b993-01be64020b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:16.626724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:16.693058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 539/1064] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:34:16.989970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399f75d-e559-46f4-a630-2d59aa47b83f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:16.991519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0399f75d-e559-46f4-a630-2d59aa47b83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:17.067805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:17.141215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399f75d-e559-46f4-a630-2d59aa47b83f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:17.267042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144ca0d-46d5-4ff6-93b4-fd9f834df2f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:17.267712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144ca0d-46d5-4ff6-93b4-fd9f834df2f2 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:17.268075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144ca0d-46d5-4ff6-93b4-fd9f834df2f2 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:17.268517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144ca0d-46d5-4ff6-93b4-fd9f834df2f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:17.270592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0144ca0d-46d5-4ff6-93b4-fd9f834df2f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:17.311518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:17.367261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144ca0d-46d5-4ff6-93b4-fd9f834df2f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:17.368908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144ca0d-46d5-4ff6-93b4-fd9f834df2f2 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:17.388983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 526/1065] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:19.726833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63eb52-11c8-4573-a073-ff42cc40f8d3 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:19.727440 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63eb52-11c8-4573-a073-ff42cc40f8d3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:19.727440 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63eb52-11c8-4573-a073-ff42cc40f8d3 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:19.728800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 540/1066] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:19.734713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42da9d-984d-4d49-b9dd-e744a2f3d9ef None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:19.735166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42da9d-984d-4d49-b9dd-e744a2f3d9ef None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:19.735652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42da9d-984d-4d49-b9dd-e744a2f3d9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:19.802031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5a42da9d-984d-4d49-b9dd-e744a2f3d9ef None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:19.803231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5a42da9d-984d-4d49-b9dd-e744a2f3d9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:19.899985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 527/1067] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:20.190900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63eb52-11c8-4573-a073-ff42cc40f8d3 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:20.192433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fa63eb52-11c8-4573-a073-ff42cc40f8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:20.233169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf700d-3836-4707-8037-a0a0b63e0413 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:20.233449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf700d-3836-4707-8037-a0a0b63e0413 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:20.233720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf700d-3836-4707-8037-a0a0b63e0413 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:20.235331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 528/1068] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:20.240122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccbb02-f700-4842-91fd-d19f54310a63 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:20.240431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccbb02-f700-4842-91fd-d19f54310a63 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:20.240709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccbb02-f700-4842-91fd-d19f54310a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:20.256707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:20.309335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-cbccbb02-f700-4842-91fd-d19f54310a63 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:20.310609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cbccbb02-f700-4842-91fd-d19f54310a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:20.345078 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63eb52-11c8-4573-a073-ff42cc40f8d3 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:20.362321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:20.413013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 529/1069] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:34:20.464763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de3d49-38a8-4909-b043-4d5085444dac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:20.465946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de3d49-38a8-4909-b043-4d5085444dac service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:20.466290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de3d49-38a8-4909-b043-4d5085444dac service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:20.466562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de3d49-38a8-4909-b043-4d5085444dac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:20.468142 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-55de3d49-38a8-4909-b043-4d5085444dac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:20.556076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de3d49-38a8-4909-b043-4d5085444dac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:20.557677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de3d49-38a8-4909-b043-4d5085444dac service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:20.575873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 541/1070] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:20.721031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e1954-32a8-4343-9bcc-14481786e5b0 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:20.721343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e1954-32a8-4343-9bcc-14481786e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:20.721686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e1954-32a8-4343-9bcc-14481786e5b0 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:20.723114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 530/1071] 10.222.0.13 () {54 vars in 962 bytes} [Thu Oct 20 14:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:20.729021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2679f-1f86-410a-8266-262b1b7ff3e2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:20.729552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2679f-1f86-410a-8266-262b1b7ff3e2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:20.729552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2679f-1f86-410a-8266-262b1b7ff3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:20.756911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:20.814397 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d7b2679f-1f86-410a-8266-262b1b7ff3e2 None None] MFA Rules not processed for user `cacfce5c9d07471cbe1713a4d6aa29f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:20.815623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d7b2679f-1f86-410a-8266-262b1b7ff3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:20.891340 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 542/1072] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Oct 20 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:20.899273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e1954-32a8-4343-9bcc-14481786e5b0 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:20.900844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-191e1954-32a8-4343-9bcc-14481786e5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:20.999076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:21.005745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e1954-32a8-4343-9bcc-14481786e5b0 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:21.081269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771fcb5-3381-464b-a3d7-a2d126e158bf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cacfce5c9d07471cbe1713a4d6aa29f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:21.081880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771fcb5-3381-464b-a3d7-a2d126e158bf service glance] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:21.082158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771fcb5-3381-464b-a3d7-a2d126e158bf service glance] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:21.082453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771fcb5-3381-464b-a3d7-a2d126e158bf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:21.084039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1771fcb5-3381-464b-a3d7-a2d126e158bf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:21.197003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771fcb5-3381-464b-a3d7-a2d126e158bf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:21.198447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771fcb5-3381-464b-a3d7-a2d126e158bf service glance] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:21.217129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 531/1073] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:21.973811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f43bbd-8c5b-475a-9697-1dd4feb23c88 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:21.974266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f43bbd-8c5b-475a-9697-1dd4feb23c88 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:21.974485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f43bbd-8c5b-475a-9697-1dd4feb23c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:22.043519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-58f43bbd-8c5b-475a-9697-1dd4feb23c88 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:22.045017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-58f43bbd-8c5b-475a-9697-1dd4feb23c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:22.102768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:22.132782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 543/1074] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:34:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:22.180387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eed87-c289-40c1-982b-6bfa38695e81 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:22.180919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eed87-c289-40c1-982b-6bfa38695e81 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:22.181023 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eed87-c289-40c1-982b-6bfa38695e81 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:22.182333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 532/1075] 10.222.0.13 () {54 vars in 943 bytes} [Thu Oct 20 14:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:22.189050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad725e87-d169-4f04-b792-164c2b385b9a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:22.189340 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad725e87-d169-4f04-b792-164c2b385b9a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:22.189618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad725e87-d169-4f04-b792-164c2b385b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:22.266890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ad725e87-d169-4f04-b792-164c2b385b9a None None] MFA Rules not processed for user `61d6c1cd010d4fa387fa64864419bc6a`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:22.268073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ad725e87-d169-4f04-b792-164c2b385b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:22.379290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 544/1076] 10.222.0.13 () {58 vars in 1040 bytes} [Thu Oct 20 14:34:22 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:22.388741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eed87-c289-40c1-982b-6bfa38695e81 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:22.390243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-468eed87-c289-40c1-982b-6bfa38695e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:22.415389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:22.526968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eed87-c289-40c1-982b-6bfa38695e81 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:22.629258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a1ed-2866-43d0-af32-c157909ba1d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:22.629859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a1ed-2866-43d0-af32-c157909ba1d5 service swift] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:22.630132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a1ed-2866-43d0-af32-c157909ba1d5 service swift] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:22.630398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a1ed-2866-43d0-af32-c157909ba1d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:22.631994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1655a1ed-2866-43d0-af32-c157909ba1d5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:22.639961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:22.720120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655a1ed-2866-43d0-af32-c157909ba1d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:22.721745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655a1ed-2866-43d0-af32-c157909ba1d5 service swift] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:22.729681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 533/1077] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:34:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:34:22.794686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce0975-955b-4b42-a9d1-b76171621f62 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:22.794989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce0975-955b-4b42-a9d1-b76171621f62 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:22.795265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce0975-955b-4b42-a9d1-b76171621f62 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:22.796688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 545/1078] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:22.802415 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aca53-ff0b-487e-88a1-027b28dcca5f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:22.802702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aca53-ff0b-487e-88a1-027b28dcca5f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:22.802978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aca53-ff0b-487e-88a1-027b28dcca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:22.883953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-150aca53-ff0b-487e-88a1-027b28dcca5f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:22.885176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-150aca53-ff0b-487e-88a1-027b28dcca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:22.994307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:22.997043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 534/1079] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:34:23.289346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce0975-955b-4b42-a9d1-b76171621f62 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:23.291137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-27ce0975-955b-4b42-a9d1-b76171621f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:23.336175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:23.464012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce0975-955b-4b42-a9d1-b76171621f62 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:23.584676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:23.586773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dac97c-74af-473f-a2a3-d8fe5baff918 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:23.587704 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dac97c-74af-473f-a2a3-d8fe5baff918 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:23.588005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dac97c-74af-473f-a2a3-d8fe5baff918 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:23.588512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dac97c-74af-473f-a2a3-d8fe5baff918 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:23.591470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-86dac97c-74af-473f-a2a3-d8fe5baff918 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:23.697497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dac97c-74af-473f-a2a3-d8fe5baff918 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:23.699244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dac97c-74af-473f-a2a3-d8fe5baff918 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:23.721208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 546/1080] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:25.942558 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf968-bdfd-4378-9d0a-14ced2bf6231 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:25.943531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf968-bdfd-4378-9d0a-14ced2bf6231 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:25.943909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf968-bdfd-4378-9d0a-14ced2bf6231 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:25.945757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 535/1081] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:25.951496 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58e53f-c842-468d-8e95-03c2150834a3 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:25.951496 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58e53f-c842-468d-8e95-03c2150834a3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:25.951947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58e53f-c842-468d-8e95-03c2150834a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:26.030705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ca58e53f-c842-468d-8e95-03c2150834a3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:26.032041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ca58e53f-c842-468d-8e95-03c2150834a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:26.107972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:26.140016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 547/1082] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:26.435049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf968-bdfd-4378-9d0a-14ced2bf6231 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:26.436607 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-adaaf968-bdfd-4378-9d0a-14ced2bf6231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:26.600845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf968-bdfd-4378-9d0a-14ced2bf6231 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:26.686684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:26.728593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561dfca-bdb4-4a2e-8a6b-537992955097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:26.729202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561dfca-bdb4-4a2e-8a6b-537992955097 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:26.729478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561dfca-bdb4-4a2e-8a6b-537992955097 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:26.729748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561dfca-bdb4-4a2e-8a6b-537992955097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:26.731493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8561dfca-bdb4-4a2e-8a6b-537992955097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:26.834132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561dfca-bdb4-4a2e-8a6b-537992955097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:26.835937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561dfca-bdb4-4a2e-8a6b-537992955097 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:26.855861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 536/1083] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:34:29.070183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e600f-602d-40de-ab51-84edb26787a6 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:29.070725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e600f-602d-40de-ab51-84edb26787a6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:29.070725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e600f-602d-40de-ab51-84edb26787a6 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:29.072203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 548/1084] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:29.077905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e273c78-df2b-432b-be65-1f999a7fa6f4 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:29.078360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e273c78-df2b-432b-be65-1f999a7fa6f4 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:29.078456 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e273c78-df2b-432b-be65-1f999a7fa6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:29.147429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5e273c78-df2b-432b-be65-1f999a7fa6f4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:29.148822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5e273c78-df2b-432b-be65-1f999a7fa6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:29.191975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:29.259267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 537/1085] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:34:29.556858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e600f-602d-40de-ab51-84edb26787a6 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:29.558472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e60e600f-602d-40de-ab51-84edb26787a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:29.738153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e600f-602d-40de-ab51-84edb26787a6 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:29.824384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:29.893080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4ce3d-bed1-46b2-8eeb-e82f9fd75563 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:29.893695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4ce3d-bed1-46b2-8eeb-e82f9fd75563 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:29.893979 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4ce3d-bed1-46b2-8eeb-e82f9fd75563 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:29.894279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4ce3d-bed1-46b2-8eeb-e82f9fd75563 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:29.895857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-83f4ce3d-bed1-46b2-8eeb-e82f9fd75563 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:29.997543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4ce3d-bed1-46b2-8eeb-e82f9fd75563 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:29.999313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4ce3d-bed1-46b2-8eeb-e82f9fd75563 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:30.018642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 549/1086] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:39.257412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=27 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:40.016354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=73 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:45.544363 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8cb24-cad0-43ef-8007-8b6fb4b45026 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:45.544917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8cb24-cad0-43ef-8007-8b6fb4b45026 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:45.544917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8cb24-cad0-43ef-8007-8b6fb4b45026 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:45.546258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 538/1087] 10.222.0.13 () {54 vars in 918 bytes} [Thu Oct 20 14:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:45.551342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedecaf6-5b0b-4b2e-a201-8676fcf7e992 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:45.551774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedecaf6-5b0b-4b2e-a201-8676fcf7e992 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:45.551870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedecaf6-5b0b-4b2e-a201-8676fcf7e992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:45.629680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-fedecaf6-5b0b-4b2e-a201-8676fcf7e992 None None] MFA Rules not processed for user `fc18a78229064e8d8ea4321c3c23c4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:45.630823 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fedecaf6-5b0b-4b2e-a201-8676fcf7e992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:45.733995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 550/1088] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Oct 20 14:34:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:34:45.748633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c8df7-2628-4c55-ab07-eb50dfcf6384 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:45.748927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c8df7-2628-4c55-ab07-eb50dfcf6384 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:45.749204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c8df7-2628-4c55-ab07-eb50dfcf6384 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:45.750553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 539/1089] 10.222.0.13 () {54 vars in 918 bytes} [Thu Oct 20 14:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:45.754990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a72520-1ee4-4b9a-b692-f8f009ba8d70 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:45.755275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a72520-1ee4-4b9a-b692-f8f009ba8d70 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:45.755553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a72520-1ee4-4b9a-b692-f8f009ba8d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:45.827726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:45.830436 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-19a72520-1ee4-4b9a-b692-f8f009ba8d70 None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:45.832193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-19a72520-1ee4-4b9a-b692-f8f009ba8d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:45.946345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 551/1090] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Oct 20 14:34:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:34:48.491127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb83ba-85fa-4d2c-aac0-9fa018a67e14 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:48.491720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb83ba-85fa-4d2c-aac0-9fa018a67e14 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:48.491720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb83ba-85fa-4d2c-aac0-9fa018a67e14 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:48.493080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 540/1091] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:48.498338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d882b-eb4e-43b9-b5b4-9b5a568583c6 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:48.498776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d882b-eb4e-43b9-b5b4-9b5a568583c6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:48.498889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d882b-eb4e-43b9-b5b4-9b5a568583c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:48.564578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-fd5d882b-eb4e-43b9-b5b4-9b5a568583c6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:48.565765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fd5d882b-eb4e-43b9-b5b4-9b5a568583c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:48.619787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:48.660613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 552/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:48.951560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb83ba-85fa-4d2c-aac0-9fa018a67e14 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:48.953258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e3cb83ba-85fa-4d2c-aac0-9fa018a67e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:49.099000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb83ba-85fa-4d2c-aac0-9fa018a67e14 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:49.170768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:49.210497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8dc92d-0e5a-4bd4-99f6-3036db405cb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:49.211089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dc92d-0e5a-4bd4-99f6-3036db405cb0 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:49.211363 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dc92d-0e5a-4bd4-99f6-3036db405cb0 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:49.211631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dc92d-0e5a-4bd4-99f6-3036db405cb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:49.213227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cb8dc92d-0e5a-4bd4-99f6-3036db405cb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:49.301070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dc92d-0e5a-4bd4-99f6-3036db405cb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:49.302665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dc92d-0e5a-4bd4-99f6-3036db405cb0 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:49.320541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 541/1093] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:50.450024 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f9255-3956-4942-b6a2-c2ef5de2fe4d None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:50.450559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f9255-3956-4942-b6a2-c2ef5de2fe4d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:50.450654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f9255-3956-4942-b6a2-c2ef5de2fe4d None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:50.452231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 553/1094] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:50.458646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a0a64-243d-4dc1-b835-40fb04ae00c6 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:50.459096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a0a64-243d-4dc1-b835-40fb04ae00c6 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:50.459191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a0a64-243d-4dc1-b835-40fb04ae00c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:50.534773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-017a0a64-243d-4dc1-b835-40fb04ae00c6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:50.535977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-017a0a64-243d-4dc1-b835-40fb04ae00c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:50.563475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:50.634086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 542/1095] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:50.784400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f9255-3956-4942-b6a2-c2ef5de2fe4d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:50.785914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7b9f9255-3956-4942-b6a2-c2ef5de2fe4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:50.928650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f9255-3956-4942-b6a2-c2ef5de2fe4d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:50.968226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:51.040720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad00ee4-a3ad-4201-bf15-b716b1db462b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:51.041327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad00ee4-a3ad-4201-bf15-b716b1db462b service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:51.041603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad00ee4-a3ad-4201-bf15-b716b1db462b service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:51.041871 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad00ee4-a3ad-4201-bf15-b716b1db462b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:51.044087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8ad00ee4-a3ad-4201-bf15-b716b1db462b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:51.130530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad00ee4-a3ad-4201-bf15-b716b1db462b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:51.132102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad00ee4-a3ad-4201-bf15-b716b1db462b service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:51.150142 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 554/1096] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:52.279062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb18255-0ee5-4ed4-83ce-31a0ead179da None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:52.279584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb18255-0ee5-4ed4-83ce-31a0ead179da None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:52.279584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb18255-0ee5-4ed4-83ce-31a0ead179da None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:52.280931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 543/1097] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:52.287180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5446f72-8518-458b-8984-19de5015ff70 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:52.287592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5446f72-8518-458b-8984-19de5015ff70 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:52.287728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5446f72-8518-458b-8984-19de5015ff70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:52.342477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:52.365652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c5446f72-8518-458b-8984-19de5015ff70 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:52.366861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c5446f72-8518-458b-8984-19de5015ff70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:52.460586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 555/1098] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:52.613596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb18255-0ee5-4ed4-83ce-31a0ead179da None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:52.615069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2bb18255-0ee5-4ed4-83ce-31a0ead179da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:52.761063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb18255-0ee5-4ed4-83ce-31a0ead179da None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:52.772880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:52.872960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4513d-bcf7-42f0-9743-f134804cc3f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:52.873479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4513d-bcf7-42f0-9743-f134804cc3f6 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:52.873754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4513d-bcf7-42f0-9743-f134804cc3f6 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:52.874025 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4513d-bcf7-42f0-9743-f134804cc3f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:52.875594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-96f4513d-bcf7-42f0-9743-f134804cc3f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:52.960941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4513d-bcf7-42f0-9743-f134804cc3f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:52.962516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4513d-bcf7-42f0-9743-f134804cc3f6 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:52.981109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 544/1099] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:54.094763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61896a1-9919-4983-9a5e-30be2c82bee2 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:54.095297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61896a1-9919-4983-9a5e-30be2c82bee2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:54.095297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61896a1-9919-4983-9a5e-30be2c82bee2 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:54.096681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 556/1100] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:54.102740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a429e9-9f64-4d78-8cf7-bd97ad0938b0 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:54.103200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a429e9-9f64-4d78-8cf7-bd97ad0938b0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:54.103299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a429e9-9f64-4d78-8cf7-bd97ad0938b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:54.126803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:54.183683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-98a429e9-9f64-4d78-8cf7-bd97ad0938b0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:54.184922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-98a429e9-9f64-4d78-8cf7-bd97ad0938b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:54.280958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 545/1101] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:54.430492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61896a1-9919-4983-9a5e-30be2c82bee2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:54.432036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d61896a1-9919-4983-9a5e-30be2c82bee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:54.541532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:54.577985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61896a1-9919-4983-9a5e-30be2c82bee2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:54.690102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dac4e-64b5-435e-9b28-a79877d89ef6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:54.690706 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dac4e-64b5-435e-9b28-a79877d89ef6 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:54.690982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dac4e-64b5-435e-9b28-a79877d89ef6 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:54.691252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dac4e-64b5-435e-9b28-a79877d89ef6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:54.692863 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7a2dac4e-64b5-435e-9b28-a79877d89ef6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:54.768861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:54.783178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2dac4e-64b5-435e-9b28-a79877d89ef6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:54.784782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2dac4e-64b5-435e-9b28-a79877d89ef6 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:54.803163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 557/1102] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:55.917990 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6a17b-2443-4c85-9eaa-cf8d16d6090d None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:55.918536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6a17b-2443-4c85-9eaa-cf8d16d6090d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:55.918536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6a17b-2443-4c85-9eaa-cf8d16d6090d None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:55.919901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 546/1103] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:55.926143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8820c-11a8-48d7-98bf-8e1976625236 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:55.926561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8820c-11a8-48d7-98bf-8e1976625236 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:55.926689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8820c-11a8-48d7-98bf-8e1976625236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:56.005601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c3e8820c-11a8-48d7-98bf-8e1976625236 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:56.006862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c3e8820c-11a8-48d7-98bf-8e1976625236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:56.102328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 558/1104] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:56.253568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6a17b-2443-4c85-9eaa-cf8d16d6090d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:56.255190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-30a6a17b-2443-4c85-9eaa-cf8d16d6090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:56.344648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:56.409923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6a17b-2443-4c85-9eaa-cf8d16d6090d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:56.520894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe2d9b-89e5-46a7-bf29-09664766ce6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:56.521396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe2d9b-89e5-46a7-bf29-09664766ce6b service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:56.521669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe2d9b-89e5-46a7-bf29-09664766ce6b service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:56.521938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe2d9b-89e5-46a7-bf29-09664766ce6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:56.523510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5cbe2d9b-89e5-46a7-bf29-09664766ce6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:56.571641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:56.612820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe2d9b-89e5-46a7-bf29-09664766ce6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:56.614412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe2d9b-89e5-46a7-bf29-09664766ce6b service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:56.632667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 547/1105] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:57.675376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4e676-8dae-4f00-856d-8509e432c038 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:57.675883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4e676-8dae-4f00-856d-8509e432c038 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:57.675883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4e676-8dae-4f00-856d-8509e432c038 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:57.677281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 559/1106] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:57.683997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb4f18-51ae-4e9c-8991-99564e36f2d3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:57.684475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb4f18-51ae-4e9c-8991-99564e36f2d3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:57.684579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb4f18-51ae-4e9c-8991-99564e36f2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:57.760116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-deeb4f18-51ae-4e9c-8991-99564e36f2d3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:57.762012 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-deeb4f18-51ae-4e9c-8991-99564e36f2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:57.855578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 548/1107] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:58.010189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4e676-8dae-4f00-856d-8509e432c038 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:58.011729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b0a4e676-8dae-4f00-856d-8509e432c038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:58.059850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:58.155039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4e676-8dae-4f00-856d-8509e432c038 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:58.266783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1bd40-4c54-4591-a14b-c44b6b68aa8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:34:58.267307 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1bd40-4c54-4591-a14b-c44b6b68aa8b service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:58.267581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1bd40-4c54-4591-a14b-c44b6b68aa8b service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:58.267851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1bd40-4c54-4591-a14b-c44b6b68aa8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:58.269461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e5f1bd40-4c54-4591-a14b-c44b6b68aa8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:58.276237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:58.358285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1bd40-4c54-4591-a14b-c44b6b68aa8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:34:58.360137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1bd40-4c54-4591-a14b-c44b6b68aa8b service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:34:58.378117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 560/1108] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:34:59.421313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9cf0d-c472-445e-ba04-0094ae6eb5e5 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:59.421842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9cf0d-c472-445e-ba04-0094ae6eb5e5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:59.421951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9cf0d-c472-445e-ba04-0094ae6eb5e5 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:59.423254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 549/1109] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:34:59.429499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b51f8c-1c6a-46de-8a4d-6bf3b1f9307b None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:34:59.429915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b51f8c-1c6a-46de-8a4d-6bf3b1f9307b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:34:59.430050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b51f8c-1c6a-46de-8a4d-6bf3b1f9307b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:34:59.505778 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-45b51f8c-1c6a-46de-8a4d-6bf3b1f9307b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:34:59.507244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-45b51f8c-1c6a-46de-8a4d-6bf3b1f9307b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:59.577943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:59.607385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 561/1110] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:34:59.757874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9cf0d-c472-445e-ba04-0094ae6eb5e5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:34:59.759404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ccc9cf0d-c472-445e-ba04-0094ae6eb5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:34:59.776273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:34:59.901971 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9cf0d-c472-445e-ba04-0094ae6eb5e5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:34:59.987916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:00.014382 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e08735-cfad-4910-b380-f74a81c2ea4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:00.014992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e08735-cfad-4910-b380-f74a81c2ea4e service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:00.015273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e08735-cfad-4910-b380-f74a81c2ea4e service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:00.015548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e08735-cfad-4910-b380-f74a81c2ea4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:00.017172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e4e08735-cfad-4910-b380-f74a81c2ea4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:00.106536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e08735-cfad-4910-b380-f74a81c2ea4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:00.108126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e08735-cfad-4910-b380-f74a81c2ea4e service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:00.126258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 550/1111] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:01.181532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba2c2a-e576-4e4a-a0d3-755a15aee790 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:01.182093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba2c2a-e576-4e4a-a0d3-755a15aee790 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:01.182093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba2c2a-e576-4e4a-a0d3-755a15aee790 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:01.183978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 562/1112] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:01.190159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eb589-63a0-4d0b-a4d8-8790995e841e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:01.190604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eb589-63a0-4d0b-a4d8-8790995e841e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:01.190721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eb589-63a0-4d0b-a4d8-8790995e841e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:01.272007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b80eb589-63a0-4d0b-a4d8-8790995e841e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:01.273246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b80eb589-63a0-4d0b-a4d8-8790995e841e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:01.315071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:01.369652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 551/1113] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:01.525475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba2c2a-e576-4e4a-a0d3-755a15aee790 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:01.527906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-38ba2c2a-e576-4e4a-a0d3-755a15aee790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:01.673644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba2c2a-e576-4e4a-a0d3-755a15aee790 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:01.726414 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:01.783902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86157f-e9df-46df-adb8-3bd7155b8f5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:01.784452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86157f-e9df-46df-adb8-3bd7155b8f5a service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:01.784732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86157f-e9df-46df-adb8-3bd7155b8f5a service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:01.785003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86157f-e9df-46df-adb8-3bd7155b8f5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:01.786577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3b86157f-e9df-46df-adb8-3bd7155b8f5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:01.874387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86157f-e9df-46df-adb8-3bd7155b8f5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:01.875962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86157f-e9df-46df-adb8-3bd7155b8f5a service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:01.893098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 563/1114] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:02.973019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470778c-6589-4ac2-a64b-8760cebc7071 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:02.973598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470778c-6589-4ac2-a64b-8760cebc7071 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:02.973598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470778c-6589-4ac2-a64b-8760cebc7071 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:02.975013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 552/1115] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:02.982278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4b542-6f00-489b-b3d5-3676c42fb449 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:02.982789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4b542-6f00-489b-b3d5-3676c42fb449 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:02.982904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4b542-6f00-489b-b3d5-3676c42fb449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:03.067861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-cff4b542-6f00-489b-b3d5-3676c42fb449 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:03.069089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-cff4b542-6f00-489b-b3d5-3676c42fb449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:03.071799 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:03.180878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 564/1116] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:03.346433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470778c-6589-4ac2-a64b-8760cebc7071 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:03.347893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8470778c-6589-4ac2-a64b-8760cebc7071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:03.497471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470778c-6589-4ac2-a64b-8760cebc7071 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:03.523875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:03.611139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48f20d-b1ed-447b-b8a8-163553f404a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:03.611656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f20d-b1ed-447b-b8a8-163553f404a9 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:03.611932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f20d-b1ed-447b-b8a8-163553f404a9 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:03.612202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f20d-b1ed-447b-b8a8-163553f404a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:03.613833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5c48f20d-b1ed-447b-b8a8-163553f404a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:03.703848 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f20d-b1ed-447b-b8a8-163553f404a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:03.705465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f20d-b1ed-447b-b8a8-163553f404a9 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:03.722830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 553/1117] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:05.786044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac18a3f-220c-4011-950a-10fd19f849f4 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:05.786608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac18a3f-220c-4011-950a-10fd19f849f4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:05.786608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac18a3f-220c-4011-950a-10fd19f849f4 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:05.787913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 565/1118] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:05.793535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dacbb5-4769-4dce-909f-0d22e42ee2f8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:05.793987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dacbb5-4769-4dce-909f-0d22e42ee2f8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:05.794079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dacbb5-4769-4dce-909f-0d22e42ee2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:05.840734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:05.865798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-87dacbb5-4769-4dce-909f-0d22e42ee2f8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:05.866995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-87dacbb5-4769-4dce-909f-0d22e42ee2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:05.962408 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 554/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:06.253426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac18a3f-220c-4011-950a-10fd19f849f4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:06.254987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6ac18a3f-220c-4011-950a-10fd19f849f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:06.357742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac18a3f-220c-4011-950a-10fd19f849f4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:06.389828 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:06.430167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ea9d7-e983-4249-b75b-63464a85b6ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:06.430763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ea9d7-e983-4249-b75b-63464a85b6ad service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:06.431047 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ea9d7-e983-4249-b75b-63464a85b6ad service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:06.431320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ea9d7-e983-4249-b75b-63464a85b6ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:06.433450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d70ea9d7-e983-4249-b75b-63464a85b6ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:06.545624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ea9d7-e983-4249-b75b-63464a85b6ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:06.546979 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ea9d7-e983-4249-b75b-63464a85b6ad service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:06.565301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 566/1120] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:08.871479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3855-5ccd-43d4-a6e3-1d9deda6987f None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:08.872043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3855-5ccd-43d4-a6e3-1d9deda6987f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:08.872043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3855-5ccd-43d4-a6e3-1d9deda6987f None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:08.873415 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 555/1121] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:08.878950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a896ec-50e5-452d-baeb-60f92faf50e9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:08.879332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a896ec-50e5-452d-baeb-60f92faf50e9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:08.879488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a896ec-50e5-452d-baeb-60f92faf50e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:08.945175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-92a896ec-50e5-452d-baeb-60f92faf50e9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:08.946365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-92a896ec-50e5-452d-baeb-60f92faf50e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:08.958862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:09.042676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 567/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:09.323272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e3855-5ccd-43d4-a6e3-1d9deda6987f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:09.324936 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2a9e3855-5ccd-43d4-a6e3-1d9deda6987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:09.428128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e3855-5ccd-43d4-a6e3-1d9deda6987f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:09.441079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:09.502804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff95ed1-b328-4947-ae99-44f0a8ee6536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:09.503403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff95ed1-b328-4947-ae99-44f0a8ee6536 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:09.503681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff95ed1-b328-4947-ae99-44f0a8ee6536 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:09.503961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff95ed1-b328-4947-ae99-44f0a8ee6536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:09.505580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4ff95ed1-b328-4947-ae99-44f0a8ee6536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:09.615464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff95ed1-b328-4947-ae99-44f0a8ee6536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:09.616861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff95ed1-b328-4947-ae99-44f0a8ee6536 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:09.635271 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 556/1123] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:12.178339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd24b2-8c15-4368-a221-c00bfcce7a08 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:12.178892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd24b2-8c15-4368-a221-c00bfcce7a08 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:12.179004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd24b2-8c15-4368-a221-c00bfcce7a08 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:12.180319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 568/1124] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:12.186480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f010d-24b3-4ece-81f1-b98d45d9eec5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:12.186905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f010d-24b3-4ece-81f1-b98d45d9eec5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:12.187042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f010d-24b3-4ece-81f1-b98d45d9eec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:12.234102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:12.253267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-9b7f010d-24b3-4ece-81f1-b98d45d9eec5 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:12.254472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9b7f010d-24b3-4ece-81f1-b98d45d9eec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:12.348761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 557/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:12.635652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd24b2-8c15-4368-a221-c00bfcce7a08 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:12.637392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-75dd24b2-8c15-4368-a221-c00bfcce7a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:12.740911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd24b2-8c15-4368-a221-c00bfcce7a08 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:12.779983 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:12.813989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e685-8119-4a7d-8771-f64146e8c7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:12.814590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e685-8119-4a7d-8771-f64146e8c7ad service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:12.814862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e685-8119-4a7d-8771-f64146e8c7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:12.815130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e685-8119-4a7d-8771-f64146e8c7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:12.816731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0039e685-8119-4a7d-8771-f64146e8c7ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:12.924234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e685-8119-4a7d-8771-f64146e8c7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:12.925601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e685-8119-4a7d-8771-f64146e8c7ad service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:12.943164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 569/1126] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:15.232049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5bafa-4a99-4a6c-a539-099a67946358 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:15.232626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5bafa-4a99-4a6c-a539-099a67946358 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:15.232626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5bafa-4a99-4a6c-a539-099a67946358 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:15.234008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 558/1127] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:15.239622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2cf98-bb98-4f61-b31e-2b7318a0a06c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:15.240016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2cf98-bb98-4f61-b31e-2b7318a0a06c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:15.240172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2cf98-bb98-4f61-b31e-2b7318a0a06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:15.310203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-eda2cf98-bb98-4f61-b31e-2b7318a0a06c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:15.311423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-eda2cf98-bb98-4f61-b31e-2b7318a0a06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:15.331481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:15.409346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 570/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:15.700302 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5bafa-4a99-4a6c-a539-099a67946358 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:15.701848 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-70a5bafa-4a99-4a6c-a539-099a67946358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:15.804235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5bafa-4a99-4a6c-a539-099a67946358 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:15.824999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:15.878311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18f07f-46e5-40dd-877a-8c259a807417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:15.878905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18f07f-46e5-40dd-877a-8c259a807417 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:15.879184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18f07f-46e5-40dd-877a-8c259a807417 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:15.879455 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18f07f-46e5-40dd-877a-8c259a807417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:15.881060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0d18f07f-46e5-40dd-877a-8c259a807417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:15.987853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18f07f-46e5-40dd-877a-8c259a807417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:15.989323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18f07f-46e5-40dd-877a-8c259a807417 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:16.008054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 559/1129] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:18.191555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28f46-d7a4-43cf-a9d2-158601814148 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:18.192104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28f46-d7a4-43cf-a9d2-158601814148 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:18.192104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28f46-d7a4-43cf-a9d2-158601814148 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:18.193491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 571/1130] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:18.199549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea82eb-3f1b-4bb0-8d49-33ab0760804e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:18.199976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea82eb-3f1b-4bb0-8d49-33ab0760804e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:18.200103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea82eb-3f1b-4bb0-8d49-33ab0760804e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:18.264170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:18.267443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-79ea82eb-3f1b-4bb0-8d49-33ab0760804e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:18.269325 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-79ea82eb-3f1b-4bb0-8d49-33ab0760804e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:18.366386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 560/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:18.655092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f28f46-d7a4-43cf-a9d2-158601814148 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:18.656887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f1f28f46-d7a4-43cf-a9d2-158601814148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:18.758714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f28f46-d7a4-43cf-a9d2-158601814148 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:18.812946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:18.836843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bcb02-a7e6-404e-ab01-745f0888a4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:18.837451 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bcb02-a7e6-404e-ab01-745f0888a4bd service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:18.837727 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bcb02-a7e6-404e-ab01-745f0888a4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:18.837998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bcb02-a7e6-404e-ab01-745f0888a4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:18.839573 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bb1bcb02-a7e6-404e-ab01-745f0888a4bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:18.948128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bcb02-a7e6-404e-ab01-745f0888a4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:18.949832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bcb02-a7e6-404e-ab01-745f0888a4bd service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:18.967792 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 572/1132] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:21.221934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464c94e-b87c-43ad-b10d-12e180f42f26 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:21.222502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464c94e-b87c-43ad-b10d-12e180f42f26 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:21.222502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464c94e-b87c-43ad-b10d-12e180f42f26 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:21.223842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 561/1133] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:21.229208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc2954-6cf3-4534-8155-5d95c36e5907 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:21.229676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc2954-6cf3-4534-8155-5d95c36e5907 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:21.229785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc2954-6cf3-4534-8155-5d95c36e5907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:21.296797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-92fc2954-6cf3-4534-8155-5d95c36e5907 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:21.298014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-92fc2954-6cf3-4534-8155-5d95c36e5907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:21.327138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:21.394294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 573/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:21.677347 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464c94e-b87c-43ad-b10d-12e180f42f26 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:21.678965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c464c94e-b87c-43ad-b10d-12e180f42f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:21.783832 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464c94e-b87c-43ad-b10d-12e180f42f26 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:21.812372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:21.855655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc7ddd-1517-4b44-8d57-0f4b47b392c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:21.856772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc7ddd-1517-4b44-8d57-0f4b47b392c4 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:21.857049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc7ddd-1517-4b44-8d57-0f4b47b392c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:21.857328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc7ddd-1517-4b44-8d57-0f4b47b392c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:21.858886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-efdc7ddd-1517-4b44-8d57-0f4b47b392c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:21.963649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc7ddd-1517-4b44-8d57-0f4b47b392c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:21.965193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc7ddd-1517-4b44-8d57-0f4b47b392c4 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:21.982942 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 562/1135] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:24.178117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0633cf1-c882-4b0d-a946-9e090701be71 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:24.178652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0633cf1-c882-4b0d-a946-9e090701be71 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:24.178652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0633cf1-c882-4b0d-a946-9e090701be71 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:24.179986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 574/1136] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:24.185416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b289c0-6e37-45a5-856e-043a718dc8da None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:24.185880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b289c0-6e37-45a5-856e-043a718dc8da None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:24.185977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b289c0-6e37-45a5-856e-043a718dc8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:24.251876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-95b289c0-6e37-45a5-856e-043a718dc8da None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:24.253283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-95b289c0-6e37-45a5-856e-043a718dc8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:24.261323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:24.349608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 563/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:24.633166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0633cf1-c882-4b0d-a946-9e090701be71 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:24.634748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f0633cf1-c882-4b0d-a946-9e090701be71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:24.736815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0633cf1-c882-4b0d-a946-9e090701be71 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:24.790135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:24.810122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043150e3-c1e0-4083-99f8-80c7b3df788f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:24.810727 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043150e3-c1e0-4083-99f8-80c7b3df788f service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:24.811006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043150e3-c1e0-4083-99f8-80c7b3df788f service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:24.811279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043150e3-c1e0-4083-99f8-80c7b3df788f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:24.812881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-043150e3-c1e0-4083-99f8-80c7b3df788f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:24.919898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043150e3-c1e0-4083-99f8-80c7b3df788f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:24.921280 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043150e3-c1e0-4083-99f8-80c7b3df788f service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:24.939911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 575/1138] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:27.149139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37f421-d4e4-4bc9-9833-ca9c889f6fd2 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:27.149683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37f421-d4e4-4bc9-9833-ca9c889f6fd2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:27.149683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37f421-d4e4-4bc9-9833-ca9c889f6fd2 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:27.151010 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 564/1139] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:27.157194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697cc13-92a2-4c87-8224-c94e41861b96 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:27.157601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697cc13-92a2-4c87-8224-c94e41861b96 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:27.157742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697cc13-92a2-4c87-8224-c94e41861b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:27.223838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-a697cc13-92a2-4c87-8224-c94e41861b96 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:27.225066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a697cc13-92a2-4c87-8224-c94e41861b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:27.262848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:27.320853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 576/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:27.615109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37f421-d4e4-4bc9-9833-ca9c889f6fd2 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:27.616674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7d37f421-d4e4-4bc9-9833-ca9c889f6fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:27.719532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37f421-d4e4-4bc9-9833-ca9c889f6fd2 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:27.757299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:27.791670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8adfe6-da1d-467f-baa2-9df8149bfc0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:27.792262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8adfe6-da1d-467f-baa2-9df8149bfc0a service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:27.792566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8adfe6-da1d-467f-baa2-9df8149bfc0a service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:27.792835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8adfe6-da1d-467f-baa2-9df8149bfc0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:27.794428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0a8adfe6-da1d-467f-baa2-9df8149bfc0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:27.899216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8adfe6-da1d-467f-baa2-9df8149bfc0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:27.900595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8adfe6-da1d-467f-baa2-9df8149bfc0a service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:27.917803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 565/1141] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:30.131248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eec4f6-44b8-408c-851c-728c1c0c671e None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:30.131824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eec4f6-44b8-408c-851c-728c1c0c671e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:30.131824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eec4f6-44b8-408c-851c-728c1c0c671e None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:30.133345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 577/1142] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:30.138397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2abe4-bb9d-4406-a7b7-216a8135c134 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:30.139291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2abe4-bb9d-4406-a7b7-216a8135c134 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:30.139464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2abe4-bb9d-4406-a7b7-216a8135c134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:30.204752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d2f2abe4-bb9d-4406-a7b7-216a8135c134 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:30.205922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d2f2abe4-bb9d-4406-a7b7-216a8135c134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:30.223907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:30.301359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 566/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:30.582938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eec4f6-44b8-408c-851c-728c1c0c671e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:30.584548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-25eec4f6-44b8-408c-851c-728c1c0c671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:30.686328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eec4f6-44b8-408c-851c-728c1c0c671e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:30.751834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:30.758640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621891c-2a81-478b-a93b-4306e98c3701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:30.759250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621891c-2a81-478b-a93b-4306e98c3701 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:30.759532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621891c-2a81-478b-a93b-4306e98c3701 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:30.759809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621891c-2a81-478b-a93b-4306e98c3701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:30.761427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4621891c-2a81-478b-a93b-4306e98c3701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:30.867990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621891c-2a81-478b-a93b-4306e98c3701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:30.869361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621891c-2a81-478b-a93b-4306e98c3701 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:30.887525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 578/1144] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:33.024834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b6ff-5b04-4572-827f-f660edc5b120 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:33.025427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b6ff-5b04-4572-827f-f660edc5b120 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:33.025427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b6ff-5b04-4572-827f-f660edc5b120 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:33.026729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 567/1145] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:33.032608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be2254-4ce8-4ccb-8a54-1dadecc8c750 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:33.033031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be2254-4ce8-4ccb-8a54-1dadecc8c750 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:33.033158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be2254-4ce8-4ccb-8a54-1dadecc8c750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:33.097719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d3be2254-4ce8-4ccb-8a54-1dadecc8c750 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:33.098902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d3be2254-4ce8-4ccb-8a54-1dadecc8c750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:33.146683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:33.193463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 579/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:33.478670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b6ff-5b04-4572-827f-f660edc5b120 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:33.480209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5bb6b6ff-5b04-4572-827f-f660edc5b120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:33.582300 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b6ff-5b04-4572-827f-f660edc5b120 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:33.630338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:33.656478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5190c4-984a-4b3b-8624-dc4958fe5e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:33.657074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5190c4-984a-4b3b-8624-dc4958fe5e26 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:33.657350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5190c4-984a-4b3b-8624-dc4958fe5e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:33.657621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5190c4-984a-4b3b-8624-dc4958fe5e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:33.659186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7c5190c4-984a-4b3b-8624-dc4958fe5e26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:33.766278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5190c4-984a-4b3b-8624-dc4958fe5e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:33.767716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5190c4-984a-4b3b-8624-dc4958fe5e26 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:33.785156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 568/1147] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:43.191941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=22 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:43.782825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=66 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:53.811695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660222c8-ad95-44a8-88f5-3cbaea611b09 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:53.812369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660222c8-ad95-44a8-88f5-3cbaea611b09 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:53.812369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660222c8-ad95-44a8-88f5-3cbaea611b09 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:53.813620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 580/1148] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:53.818991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c4bfb-2edc-4eee-a9c0-b6ffd50cb5a2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:53.819409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c4bfb-2edc-4eee-a9c0-b6ffd50cb5a2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:53.819546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c4bfb-2edc-4eee-a9c0-b6ffd50cb5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:53.885359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-aa9c4bfb-2edc-4eee-a9c0-b6ffd50cb5a2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:53.886482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-aa9c4bfb-2edc-4eee-a9c0-b6ffd50cb5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:53.981351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 569/1149] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:54.287346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d0242-76a1-468a-b778-27b0759262f9 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:54.287633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d0242-76a1-468a-b778-27b0759262f9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:54.287910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d0242-76a1-468a-b778-27b0759262f9 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:54.289266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 581/1150] 10.222.0.13 () {54 vars in 962 bytes} [Thu Oct 20 14:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:54.294070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b589155-c1df-461e-b89e-719a13efa1dc None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:54.294379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b589155-c1df-461e-b89e-719a13efa1dc None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:54.294744 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b589155-c1df-461e-b89e-719a13efa1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:54.370457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-2b589155-c1df-461e-b89e-719a13efa1dc None None] MFA Rules not processed for user `cacfce5c9d07471cbe1713a4d6aa29f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:54.371653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2b589155-c1df-461e-b89e-719a13efa1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:54.392690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:54.446248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 570/1151] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Oct 20 14:35:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:35:54.454923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d0242-76a1-468a-b778-27b0759262f9 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:54.456445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ac3d0242-76a1-468a-b778-27b0759262f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:54.555857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d0242-76a1-468a-b778-27b0759262f9 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:54.628393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cef0a-e9fb-4ad8-b8c4-105a37575a28 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cacfce5c9d07471cbe1713a4d6aa29f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:54.629000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cef0a-e9fb-4ad8-b8c4-105a37575a28 service glance] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:54.629279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cef0a-e9fb-4ad8-b8c4-105a37575a28 service glance] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:54.629554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cef0a-e9fb-4ad8-b8c4-105a37575a28 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:54.631142 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4a4cef0a-e9fb-4ad8-b8c4-105a37575a28 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:54.688848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:54.743554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cef0a-e9fb-4ad8-b8c4-105a37575a28 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:54.745025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cef0a-e9fb-4ad8-b8c4-105a37575a28 service glance] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:54.764207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 582/1152] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:35:56.990135 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6bb96-842d-4e52-95b3-ce08ee19a749 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:56.990711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6bb96-842d-4e52-95b3-ce08ee19a749 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:56.990711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6bb96-842d-4e52-95b3-ce08ee19a749 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:56.992059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 571/1153] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:56.998319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c66853-ac5c-4be1-ac03-6020a04e540f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:56.998769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c66853-ac5c-4be1-ac03-6020a04e540f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:56.998879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c66853-ac5c-4be1-ac03-6020a04e540f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:57.062932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-78c66853-ac5c-4be1-ac03-6020a04e540f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:57.064132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-78c66853-ac5c-4be1-ac03-6020a04e540f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:57.158319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 583/1154] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:35:57.434487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6bb96-842d-4e52-95b3-ce08ee19a749 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:35:57.436278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3cd6bb96-842d-4e52-95b3-ce08ee19a749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:35:57.559490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6bb96-842d-4e52-95b3-ce08ee19a749 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:35:57.607923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:35:57.652625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a9489-0f1c-4f5a-b1bd-39d1a1bafd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:35:57.653230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a9489-0f1c-4f5a-b1bd-39d1a1bafd12 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:57.653510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a9489-0f1c-4f5a-b1bd-39d1a1bafd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:57.653784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a9489-0f1c-4f5a-b1bd-39d1a1bafd12 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:57.654491 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a9489-0f1c-4f5a-b1bd-39d1a1bafd12 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:35:57.655603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING py.warnings [None req-dd4a9489-0f1c-4f5a-b1bd-39d1a1bafd12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:35:57.655603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: warnings.warn(msg) Oct 20 14:35:57.655603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: Oct 20 14:35:57.656778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a9489-0f1c-4f5a-b1bd-39d1a1bafd12 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:35:57.675338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 572/1155] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Oct 20 14:35:57 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:35:59.887685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de691644-6335-4ecb-a048-4f5277c50653 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:59.888281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de691644-6335-4ecb-a048-4f5277c50653 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:59.888281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de691644-6335-4ecb-a048-4f5277c50653 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:59.890429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 584/1156] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:35:59.896125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74119e1-880d-46bd-a48a-269f55de2dd1 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:35:59.896592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74119e1-880d-46bd-a48a-269f55de2dd1 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:35:59.896716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74119e1-880d-46bd-a48a-269f55de2dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:35:59.963987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b74119e1-880d-46bd-a48a-269f55de2dd1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:35:59.965213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b74119e1-880d-46bd-a48a-269f55de2dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:00.063591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 573/1157] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:36:00.358729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de691644-6335-4ecb-a048-4f5277c50653 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:36:00.360657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-de691644-6335-4ecb-a048-4f5277c50653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:00.370205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:00.502963 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de691644-6335-4ecb-a048-4f5277c50653 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:36:00.580903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:00.614451 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47276d76-ca0b-4420-af36-705fdeeb336d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:36:00.614988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47276d76-ca0b-4420-af36-705fdeeb336d service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:00.615268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47276d76-ca0b-4420-af36-705fdeeb336d service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:00.615544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47276d76-ca0b-4420-af36-705fdeeb336d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:00.617174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-47276d76-ca0b-4420-af36-705fdeeb336d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:00.704981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47276d76-ca0b-4420-af36-705fdeeb336d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:36:00.706711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47276d76-ca0b-4420-af36-705fdeeb336d service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:36:00.725066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 585/1158] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:36:03.048174 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e6728-bd07-44fd-a85b-97f88195359b None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:03.048751 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e6728-bd07-44fd-a85b-97f88195359b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:03.048869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e6728-bd07-44fd-a85b-97f88195359b None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:03.050118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 574/1159] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:36:03.056785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b023ef2-dd49-4282-8727-4a2815be00ec None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:03.057211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b023ef2-dd49-4282-8727-4a2815be00ec None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:03.057322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b023ef2-dd49-4282-8727-4a2815be00ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:03.130822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-8b023ef2-dd49-4282-8727-4a2815be00ec None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:36:03.132026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8b023ef2-dd49-4282-8727-4a2815be00ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:03.185777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:03.241884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 586/1160] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:36:03.546654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e6728-bd07-44fd-a85b-97f88195359b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:36:03.548188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f28e6728-bd07-44fd-a85b-97f88195359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:03.572271 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:03.654870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e6728-bd07-44fd-a85b-97f88195359b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:36:03.727527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c590622-bb84-44d7-a5b5-a521ce63f234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:36:03.728128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c590622-bb84-44d7-a5b5-a521ce63f234 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:03.728436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c590622-bb84-44d7-a5b5-a521ce63f234 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:03.728714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c590622-bb84-44d7-a5b5-a521ce63f234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:03.730313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7c590622-bb84-44d7-a5b5-a521ce63f234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:03.804162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:03.838610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c590622-bb84-44d7-a5b5-a521ce63f234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:36:03.840034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c590622-bb84-44d7-a5b5-a521ce63f234 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:36:03.859728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 575/1161] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:36:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:36:06.086859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a93d5f-75e8-49e8-8574-bbedafbacb8a None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:06.087369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a93d5f-75e8-49e8-8574-bbedafbacb8a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:06.087483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a93d5f-75e8-49e8-8574-bbedafbacb8a None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:06.088774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 587/1162] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:36:06.094595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d9c28-ca5f-48ef-b24b-f1e672161b93 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:06.095044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d9c28-ca5f-48ef-b24b-f1e672161b93 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:06.095166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d9c28-ca5f-48ef-b24b-f1e672161b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:06.159957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-123d9c28-ca5f-48ef-b24b-f1e672161b93 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:36:06.161254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-123d9c28-ca5f-48ef-b24b-f1e672161b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:06.256965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 576/1163] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:36:06.541900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a93d5f-75e8-49e8-8574-bbedafbacb8a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:36:06.543431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e1a93d5f-75e8-49e8-8574-bbedafbacb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:06.645060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a93d5f-75e8-49e8-8574-bbedafbacb8a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:36:06.716122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:06.717832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1c506-7168-4740-b444-6f93829a4e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:36:06.718507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1c506-7168-4740-b444-6f93829a4e3c service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:06.718785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1c506-7168-4740-b444-6f93829a4e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:06.719087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1c506-7168-4740-b444-6f93829a4e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:06.721372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-aab1c506-7168-4740-b444-6f93829a4e3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:06.828674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1c506-7168-4740-b444-6f93829a4e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:36:06.830028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1c506-7168-4740-b444-6f93829a4e3c service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:36:06.847196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 588/1164] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:36:09.188077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece23865-f45a-412a-bbd3-aa67f30689ff None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:09.188705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece23865-f45a-412a-bbd3-aa67f30689ff None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:09.188705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece23865-f45a-412a-bbd3-aa67f30689ff None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:09.189997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 577/1165] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:36:09.195417 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52641d8-8d0d-46b8-9ee6-974945f2a785 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:09.195834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52641d8-8d0d-46b8-9ee6-974945f2a785 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:09.195996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52641d8-8d0d-46b8-9ee6-974945f2a785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:09.261275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c52641d8-8d0d-46b8-9ee6-974945f2a785 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:36:09.262464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c52641d8-8d0d-46b8-9ee6-974945f2a785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:09.316281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:09.362151 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 589/1166] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:36:09.651536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece23865-f45a-412a-bbd3-aa67f30689ff None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:36:09.653131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ece23865-f45a-412a-bbd3-aa67f30689ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:09.689313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:09.756183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece23865-f45a-412a-bbd3-aa67f30689ff None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:36:09.828001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df6df0-e613-467a-bddb-d5797d47d232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:36:09.828625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6df0-e613-467a-bddb-d5797d47d232 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:09.828905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6df0-e613-467a-bddb-d5797d47d232 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:09.829174 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6df0-e613-467a-bddb-d5797d47d232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:09.830739 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-21df6df0-e613-467a-bddb-d5797d47d232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:09.910039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:09.939230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df6df0-e613-467a-bddb-d5797d47d232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:36:09.940704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df6df0-e613-467a-bddb-d5797d47d232 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:36:09.959022 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 578/1167] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:36:12.305515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5ae50-068f-49f4-adcb-e6c0a43e0730 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:12.306043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5ae50-068f-49f4-adcb-e6c0a43e0730 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:12.306043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5ae50-068f-49f4-adcb-e6c0a43e0730 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:12.307338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 590/1168] 10.222.0.13 () {54 vars in 927 bytes} [Thu Oct 20 14:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:36:12.313279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3bb1b-b587-401e-a913-f5d33826784a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:12.313686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3bb1b-b587-401e-a913-f5d33826784a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:12.313829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3bb1b-b587-401e-a913-f5d33826784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:12.380630 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-46b3bb1b-b587-401e-a913-f5d33826784a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:36:12.381813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-46b3bb1b-b587-401e-a913-f5d33826784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:12.474547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 579/1169] 10.222.0.13 () {58 vars in 1024 bytes} [Thu Oct 20 14:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:36:12.765638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5ae50-068f-49f4-adcb-e6c0a43e0730 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:36:12.767180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-67e5ae50-068f-49f4-adcb-e6c0a43e0730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:12.870591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5ae50-068f-49f4-adcb-e6c0a43e0730 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:36:12.942485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba25634-7685-49cd-adc8-c7ceafbeb1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:36:12.943079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba25634-7685-49cd-adc8-c7ceafbeb1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:36:12.943352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba25634-7685-49cd-adc8-c7ceafbeb1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:36:12.943622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba25634-7685-49cd-adc8-c7ceafbeb1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:36:12.945220 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7ba25634-7685-49cd-adc8-c7ceafbeb1c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:36:12.954833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:13.059695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba25634-7685-49cd-adc8-c7ceafbeb1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:36:13.061756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba25634-7685-49cd-adc8-c7ceafbeb1c7 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:36:13.079924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 591/1170] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:36:22.473029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=82 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:36:23.077749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=50 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:25.929920 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4dd33-3150-4e16-a0af-17cf9d4d2a51 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:25.930552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4dd33-3150-4e16-a0af-17cf9d4d2a51 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:25.930552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4dd33-3150-4e16-a0af-17cf9d4d2a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:26.005130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-47e4dd33-3150-4e16-a0af-17cf9d4d2a51 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:37:26.006331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-47e4dd33-3150-4e16-a0af-17cf9d4d2a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:26.100604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 580/1171] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:37:26.108680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf03fe5-be04-4c5c-bf77-c17a05b75493 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:26.109154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf03fe5-be04-4c5c-bf77-c17a05b75493 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:26.109255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf03fe5-be04-4c5c-bf77-c17a05b75493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:26.185990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9cf03fe5-be04-4c5c-bf77-c17a05b75493 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:37:26.187105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9cf03fe5-be04-4c5c-bf77-c17a05b75493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:26.285339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 592/1172] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:37:26.290113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4dd33-3150-4e16-a0af-17cf9d4d2a51 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:26.291321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-47e4dd33-3150-4e16-a0af-17cf9d4d2a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:26.303225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4dd33-3150-4e16-a0af-17cf9d4d2a51 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:26.367704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:26.401699 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:26.402300 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:26.402577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:26.402849 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:26.404100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:37:26.404436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:26.405447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING py.warnings [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:37:26.405447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: warnings.warn(msg) Oct 20 14:37:26.405447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: Oct 20 14:37:26.406315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:26.417922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 581/1173] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:37:26.427441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf03fe5-be04-4c5c-bf77-c17a05b75493 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:26.436476 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf03fe5-be04-4c5c-bf77-c17a05b75493 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:26.498604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:26.532926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:26.533520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:26.533793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:26.534058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:26.534710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:26.535802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING py.warnings [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:37:26.535802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: warnings.warn(msg) Oct 20 14:37:26.535802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: Oct 20 14:37:26.537024 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:26.590784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 593/1174] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:37:26 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:37:26.595590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:26.597146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:26.630806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cc4c0-496e-47df-b607-a7652ab4212e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:26.721877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:26.722487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:26.722770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:26.723044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:26.723740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:26.724817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING py.warnings [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:37:26.724817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: warnings.warn(msg) Oct 20 14:37:26.724817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: Oct 20 14:37:26.726028 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:26.767258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 582/1175] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:37:26 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:37:26.771360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:26.772862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:26.833791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0db5-40f3-4e66-9dbd-24f610acbfcc admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:26.917402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:26.928807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a01e4-d3a0-4991-9eda-ff94d73c2bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:26.929412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a01e4-d3a0-4991-9eda-ff94d73c2bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:26.929690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a01e4-d3a0-4991-9eda-ff94d73c2bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:26.929966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a01e4-d3a0-4991-9eda-ff94d73c2bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:26.930599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a01e4-d3a0-4991-9eda-ff94d73c2bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:26.932187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a01e4-d3a0-4991-9eda-ff94d73c2bd9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:26.944201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 594/1176] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:37:26.948239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:26.949798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:26.961651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076aa48-9ec9-4625-ba45-03528a6db6b7 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:26.987047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:27.058679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1f41a-5197-4138-bb8d-8a0a9e9dbdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:27.059281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1f41a-5197-4138-bb8d-8a0a9e9dbdaf admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:27.059559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1f41a-5197-4138-bb8d-8a0a9e9dbdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:27.059830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1f41a-5197-4138-bb8d-8a0a9e9dbdaf admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:27.060494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1f41a-5197-4138-bb8d-8a0a9e9dbdaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:27.062080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1f41a-5197-4138-bb8d-8a0a9e9dbdaf admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:27.074011 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 583/1177] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:37:27.078774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a01e4-d3a0-4991-9eda-ff94d73c2bd9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:27.080301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ba5a01e4-d3a0-4991-9eda-ff94d73c2bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:27.114248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a01e4-d3a0-4991-9eda-ff94d73c2bd9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:27.206952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92cf1-328d-46d6-9ffe-ab830aa7cd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:27.207600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92cf1-328d-46d6-9ffe-ab830aa7cd74 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:27.207881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92cf1-328d-46d6-9ffe-ab830aa7cd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:27.208155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92cf1-328d-46d6-9ffe-ab830aa7cd74 admin admin] PATH_INFO: `/v3/projects/b277bd8e6c774429a2a6f4ae85462c7d/users/8e6fc658a9e14c048522089c5408c7f2/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:27.231275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f92cf1-328d-46d6-9ffe-ab830aa7cd74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b277bd8e6c774429a2a6f4ae85462c7d, user_id=8e6fc658a9e14c048522089c5408c7f2, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:27.233992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f92cf1-328d-46d6-9ffe-ab830aa7cd74 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:27.257716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 595/1178] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:37:27 2022] PUT /identity/v3/projects/b277bd8e6c774429a2a6f4ae85462c7d/users/8e6fc658a9e14c048522089c5408c7f2/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:37:27.269297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdc5d4-3a35-423b-9350-eaece553adb8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:27.269590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdc5d4-3a35-423b-9350-eaece553adb8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:27.269867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdc5d4-3a35-423b-9350-eaece553adb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:27.327497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-62cdc5d4-3a35-423b-9350-eaece553adb8 None None] MFA Rules not processed for user `8e6fc658a9e14c048522089c5408c7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:37:27.328741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-62cdc5d4-3a35-423b-9350-eaece553adb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:27.405340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 584/1179] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:37:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:37:27.418391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6f5cf-10cf-4d1d-98c3-a07562dbea88 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:27.418682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6f5cf-10cf-4d1d-98c3-a07562dbea88 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:27.418988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6f5cf-10cf-4d1d-98c3-a07562dbea88 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:27.420315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 596/1180] 10.222.0.13 () {54 vars in 963 bytes} [Thu Oct 20 14:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:37:27.425061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d648-d4ed-46e5-a0c9-fba1cb219683 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:27.425346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d648-d4ed-46e5-a0c9-fba1cb219683 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:27.425621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d648-d4ed-46e5-a0c9-fba1cb219683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:27.457306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:27.503099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b617d648-d4ed-46e5-a0c9-fba1cb219683 None None] MFA Rules not processed for user `70742ba8f3f6461f8f578b323afea7cc`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:37:27.504375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b617d648-d4ed-46e5-a0c9-fba1cb219683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:27.580108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 585/1181] 10.222.0.13 () {58 vars in 1060 bytes} [Thu Oct 20 14:37:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:37:27.587775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6f5cf-10cf-4d1d-98c3-a07562dbea88 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:27.589346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3fc6f5cf-10cf-4d1d-98c3-a07562dbea88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:27.639708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:27.693935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6f5cf-10cf-4d1d-98c3-a07562dbea88 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:27.766259 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:27.766852 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:27.767124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:27.767389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:27.768996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:27.867201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:27.868578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:27.871963 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:27.887909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 597/1182] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:37:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:37:27.932656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedee613-1042-4890-8387-41028d6b1573 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:27.932945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedee613-1042-4890-8387-41028d6b1573 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:27.933224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedee613-1042-4890-8387-41028d6b1573 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:27.934506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 586/1183] 10.222.0.13 () {52 vars in 830 bytes} [Thu Oct 20 14:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:37:27.939188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:27.940387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:28.040232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f673102-bb2f-49b2-ad6f-ab403edccb40 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:28.131495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:28.132085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:28.132304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:28.132605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] PATH_INFO: `/v3/users/8e6fc658a9e14c048522089c5408c7f2` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:28.145752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e6fc658a9e14c048522089c5408c7f2)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:28.146765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING py.warnings [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:37:28.146765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: warnings.warn(msg) Oct 20 14:37:28.146765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: Oct 20 14:37:28.147974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:28.157397 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:28.213728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e6fc658a9e14c048522089c5408c7f2'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:28.228316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e6fc658a9e14c048522089c5408c7f2'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:28.228914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e6fc658a9e14c048522089c5408c7f2'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:28.229339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e6fc658a9e14c048522089c5408c7f2'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:28.238936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e6fc658a9e14c048522089c5408c7f2'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:28.268560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 598/1184] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:37:27 2022] DELETE /identity/v3/users/8e6fc658a9e14c048522089c5408c7f2 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:37:28.272943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a35c-5fde-418d-9dbe-ca85170f8879 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:28.273434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a35c-5fde-418d-9dbe-ca85170f8879 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:28.273546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a35c-5fde-418d-9dbe-ca85170f8879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:28.330734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-8745a35c-5fde-418d-9dbe-ca85170f8879 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:37:28.331918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8745a35c-5fde-418d-9dbe-ca85170f8879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:28.406085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:28.418960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 587/1185] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:37:28.429274 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:28.430574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:28.530760 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f809432-d29c-4f3f-935e-b53d3c9d5f78 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:28.585046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:28.606324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f17278-2bac-4a94-92ee-8d2857542b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:28.607337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f17278-2bac-4a94-92ee-8d2857542b60 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:28.607620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f17278-2bac-4a94-92ee-8d2857542b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:28.607894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f17278-2bac-4a94-92ee-8d2857542b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:28.609524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b6f17278-2bac-4a94-92ee-8d2857542b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:28.717417 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f17278-2bac-4a94-92ee-8d2857542b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:28.718855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f17278-2bac-4a94-92ee-8d2857542b60 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:28.737841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 599/1186] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:37:29.091093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745a35c-5fde-418d-9dbe-ca85170f8879 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:37:29.092632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8745a35c-5fde-418d-9dbe-ca85170f8879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:37:29.104139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745a35c-5fde-418d-9dbe-ca85170f8879 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:37:29.202898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:37:29.203524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:37:29.203797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:37:29.204066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] PATH_INFO: `/v3/projects/b277bd8e6c774429a2a6f4ae85462c7d` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:37:29.213687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b277bd8e6c774429a2a6f4ae85462c7d)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:37:29.214705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING py.warnings [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:37:29.214705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: warnings.warn(msg) Oct 20 14:37:29.214705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: Oct 20 14:37:29.215889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:37:29.273690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b277bd8e6c774429a2a6f4ae85462c7d', 'request_id': 'req-f9828adc-22c4-4698-956e-29de32b5c43a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:29.281176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b277bd8e6c774429a2a6f4ae85462c7d', 'request_id': 'req-f9828adc-22c4-4698-956e-29de32b5c43a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:29.289774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b277bd8e6c774429a2a6f4ae85462c7d', 'request_id': 'req-f9828adc-22c4-4698-956e-29de32b5c43a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:29.319626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] The token cache is being invalidate because project b277bd8e6c774429a2a6f4ae85462c7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:37:29.319914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:29.320359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f9828adc-22c4-4698-956e-29de32b5c43a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:37:29.321849 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 588/1187] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:37:29 2022] DELETE /identity/v3/projects/b277bd8e6c774429a2a6f4ae85462c7d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:37:38.735343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=62 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:37:39.317600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:38:53.562158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8984e5-8aec-4d7d-aad3-6051db1c2b16 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:53.562746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8984e5-8aec-4d7d-aad3-6051db1c2b16 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:53.562746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8984e5-8aec-4d7d-aad3-6051db1c2b16 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:53.564064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 600/1188] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:38:53.568858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7546bea-0fb7-4c0b-8be6-803bb86fbfb2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:53.569292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7546bea-0fb7-4c0b-8be6-803bb86fbfb2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:53.569434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7546bea-0fb7-4c0b-8be6-803bb86fbfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:53.653162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d7546bea-0fb7-4c0b-8be6-803bb86fbfb2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:38:53.655223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d7546bea-0fb7-4c0b-8be6-803bb86fbfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:38:53.756751 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 589/1189] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:38:53.910486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8984e5-8aec-4d7d-aad3-6051db1c2b16 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:38:53.912621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-aa8984e5-8aec-4d7d-aad3-6051db1c2b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:38:54.034727 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8984e5-8aec-4d7d-aad3-6051db1c2b16 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:38:54.040119 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729b652-b5c3-46a5-8e0c-c4de7b914900 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:38:54.040746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729b652-b5c3-46a5-8e0c-c4de7b914900 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:54.041027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729b652-b5c3-46a5-8e0c-c4de7b914900 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:54.041301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729b652-b5c3-46a5-8e0c-c4de7b914900 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:54.049396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729b652-b5c3-46a5-8e0c-c4de7b914900 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:38:54.050977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729b652-b5c3-46a5-8e0c-c4de7b914900 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:38:54.060140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 601/1190] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:38:55.152907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4518d-b77a-44ae-9305-6e9f0eaf7507 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:55.153588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4518d-b77a-44ae-9305-6e9f0eaf7507 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:55.154082 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4518d-b77a-44ae-9305-6e9f0eaf7507 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:55.155492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 590/1191] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:38:55.161950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e11ecf-3faa-41c3-8baf-bc3a84ba0088 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:55.162426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e11ecf-3faa-41c3-8baf-bc3a84ba0088 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:55.162517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e11ecf-3faa-41c3-8baf-bc3a84ba0088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:55.182204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f1e11ecf-3faa-41c3-8baf-bc3a84ba0088 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:38:55.183408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f1e11ecf-3faa-41c3-8baf-bc3a84ba0088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:38:55.193680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 602/1192] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:38:55.347963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4518d-b77a-44ae-9305-6e9f0eaf7507 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:38:55.362793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4518d-b77a-44ae-9305-6e9f0eaf7507 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:38:55.368110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe8450-ab71-4b6e-b66d-72857f04035c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:38:55.368737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe8450-ab71-4b6e-b66d-72857f04035c service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:55.369014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe8450-ab71-4b6e-b66d-72857f04035c service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:55.369283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe8450-ab71-4b6e-b66d-72857f04035c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:55.377803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe8450-ab71-4b6e-b66d-72857f04035c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:38:55.379381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe8450-ab71-4b6e-b66d-72857f04035c service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:38:55.388667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 591/1193] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:38:55.707658 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e25301-d0c8-4579-8e83-e4e440029912 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:55.707941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e25301-d0c8-4579-8e83-e4e440029912 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:55.708217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e25301-d0c8-4579-8e83-e4e440029912 None None] PATH_INFO: `/v3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:55.709515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 603/1194] 10.222.0.36 () {54 vars in 925 bytes} [Thu Oct 20 14:38:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:38:55.715528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438df6a4-b4de-460b-925c-716a8c3d67af None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:55.715825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438df6a4-b4de-460b-925c-716a8c3d67af None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:55.716127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438df6a4-b4de-460b-925c-716a8c3d67af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:55.774727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-438df6a4-b4de-460b-925c-716a8c3d67af None None] MFA Rules not processed for user `fc18a78229064e8d8ea4321c3c23c4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:38:55.775927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-438df6a4-b4de-460b-925c-716a8c3d67af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:38:55.792474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:38:55.845414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 592/1195] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Oct 20 14:38:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:38:59.506211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9ce4-59eb-4512-a457-bfb37ad873e5 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:59.506702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9ce4-59eb-4512-a457-bfb37ad873e5 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:59.506801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9ce4-59eb-4512-a457-bfb37ad873e5 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:59.508105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 604/1196] 10.222.0.13 () {54 vars in 961 bytes} [Thu Oct 20 14:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:38:59.513204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f7b1e-1a85-4b6d-b850-21da177720b7 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:59.513638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f7b1e-1a85-4b6d-b850-21da177720b7 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:59.513783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f7b1e-1a85-4b6d-b850-21da177720b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:59.534243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5d9f7b1e-1a85-4b6d-b850-21da177720b7 None None] MFA Rules not processed for user `fc18a78229064e8d8ea4321c3c23c4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:38:59.535790 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5d9f7b1e-1a85-4b6d-b850-21da177720b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:38:59.547725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 593/1197] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Oct 20 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:38:59.555395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c9ce4-59eb-4512-a457-bfb37ad873e5 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:38:59.569539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c9ce4-59eb-4512-a457-bfb37ad873e5 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:38:59.574853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39de13-ef42-41e5-8785-8ea18cd05ce7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc18a78229064e8d8ea4321c3c23c4ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:38:59.576527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39de13-ef42-41e5-8785-8ea18cd05ce7 service placement] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:38:59.576813 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39de13-ef42-41e5-8785-8ea18cd05ce7 service placement] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:38:59.577091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39de13-ef42-41e5-8785-8ea18cd05ce7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:38:59.585238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39de13-ef42-41e5-8785-8ea18cd05ce7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:38:59.586808 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39de13-ef42-41e5-8785-8ea18cd05ce7 service placement] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:38:59.596751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 605/1198] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:38:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:39:06.862286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd537-6d75-4bb5-a6bd-915f12a390d4 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:06.862854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd537-6d75-4bb5-a6bd-915f12a390d4 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:06.863038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd537-6d75-4bb5-a6bd-915f12a390d4 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:06.865441 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 594/1199] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:39:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:39:06.874015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07f0b-bca9-4f57-a6f8-0e5bb9bc56df None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:06.874525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07f0b-bca9-4f57-a6f8-0e5bb9bc56df None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:06.874682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07f0b-bca9-4f57-a6f8-0e5bb9bc56df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:06.896317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-aaa07f0b-bca9-4f57-a6f8-0e5bb9bc56df None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:39:06.897569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-aaa07f0b-bca9-4f57-a6f8-0e5bb9bc56df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:39:06.905643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 606/1200] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:39:07.067800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd537-6d75-4bb5-a6bd-915f12a390d4 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:39:07.082779 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd537-6d75-4bb5-a6bd-915f12a390d4 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:39:07.088846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bfd9b-afe5-411b-b85b-3dddbb1b926f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:39:07.089439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bfd9b-afe5-411b-b85b-3dddbb1b926f service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:07.089717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bfd9b-afe5-411b-b85b-3dddbb1b926f service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:07.089989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bfd9b-afe5-411b-b85b-3dddbb1b926f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:07.099083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bfd9b-afe5-411b-b85b-3dddbb1b926f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:39:07.101091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bfd9b-afe5-411b-b85b-3dddbb1b926f service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:39:07.111312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 595/1201] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:39:08.214205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060d906-785b-41f4-941d-b1a653937d2d None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:08.214742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060d906-785b-41f4-941d-b1a653937d2d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:08.214742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060d906-785b-41f4-941d-b1a653937d2d None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:08.216159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 607/1202] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 14:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:39:08.222486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e35f7-09ca-476a-9b9b-5b75a7ef8f7f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:08.222930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e35f7-09ca-476a-9b9b-5b75a7ef8f7f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:08.223026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e35f7-09ca-476a-9b9b-5b75a7ef8f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:08.244277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-1a9e35f7-09ca-476a-9b9b-5b75a7ef8f7f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:39:08.245485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1a9e35f7-09ca-476a-9b9b-5b75a7ef8f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:39:08.254037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 596/1203] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 14:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:39:08.405062 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060d906-785b-41f4-941d-b1a653937d2d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:39:08.420082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060d906-785b-41f4-941d-b1a653937d2d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:39:08.425513 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378dd8f-a4f5-47ba-a5d5-83fbb6ab558e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:39:08.426105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378dd8f-a4f5-47ba-a5d5-83fbb6ab558e service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:08.426386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378dd8f-a4f5-47ba-a5d5-83fbb6ab558e service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:08.426660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378dd8f-a4f5-47ba-a5d5-83fbb6ab558e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:08.435095 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378dd8f-a4f5-47ba-a5d5-83fbb6ab558e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:39:08.436682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378dd8f-a4f5-47ba-a5d5-83fbb6ab558e service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:39:08.445853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 608/1204] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:39:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:39:12.463957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5532e5-fc4c-415f-8d59-36a5f57397d0 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:12.464551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5532e5-fc4c-415f-8d59-36a5f57397d0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:12.464551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5532e5-fc4c-415f-8d59-36a5f57397d0 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:12.466566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 597/1205] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:39:12.472033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e7ae7-9e6b-4d4f-8f8f-18add52f597f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:12.472511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e7ae7-9e6b-4d4f-8f8f-18add52f597f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:12.472629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e7ae7-9e6b-4d4f-8f8f-18add52f597f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:12.492416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-fa7e7ae7-9e6b-4d4f-8f8f-18add52f597f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:39:12.493608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fa7e7ae7-9e6b-4d4f-8f8f-18add52f597f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:39:12.501928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 609/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:39:12.787667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5532e5-fc4c-415f-8d59-36a5f57397d0 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:39:12.789434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0e5532e5-fc4c-415f-8d59-36a5f57397d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:39:12.866975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5532e5-fc4c-415f-8d59-36a5f57397d0 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:39:12.870576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04e139-f572-440f-afae-1a474a6d1e6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:39:12.871174 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04e139-f572-440f-afae-1a474a6d1e6c service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:12.872060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04e139-f572-440f-afae-1a474a6d1e6c service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:12.872602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04e139-f572-440f-afae-1a474a6d1e6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:12.881564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04e139-f572-440f-afae-1a474a6d1e6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:39:12.883001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04e139-f572-440f-afae-1a474a6d1e6c service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:39:12.892337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 598/1207] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:39:15.063182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb4b2e-9543-4dc3-ab2e-31c56a858773 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:15.063757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb4b2e-9543-4dc3-ab2e-31c56a858773 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:15.063757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb4b2e-9543-4dc3-ab2e-31c56a858773 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:15.065100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 610/1208] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:39:15.070204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d312381-ea77-4385-a305-9db8e39515b1 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:15.070592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d312381-ea77-4385-a305-9db8e39515b1 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:15.070764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d312381-ea77-4385-a305-9db8e39515b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:15.089934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-3d312381-ea77-4385-a305-9db8e39515b1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:39:15.091259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3d312381-ea77-4385-a305-9db8e39515b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:39:15.099993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 599/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:39:15.392362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb4b2e-9543-4dc3-ab2e-31c56a858773 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:39:15.404210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb4b2e-9543-4dc3-ab2e-31c56a858773 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:39:15.407784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c04bab-35ea-4a9d-b2ce-f3db195c97dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:39:15.408412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c04bab-35ea-4a9d-b2ce-f3db195c97dd service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:15.408698 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c04bab-35ea-4a9d-b2ce-f3db195c97dd service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:15.408974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c04bab-35ea-4a9d-b2ce-f3db195c97dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:15.417812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c04bab-35ea-4a9d-b2ce-f3db195c97dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:39:15.420244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c04bab-35ea-4a9d-b2ce-f3db195c97dd service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:39:15.429553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 611/1210] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:39:19.023002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f0b05-ebba-44fc-9a81-747f02606c24 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:19.023588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f0b05-ebba-44fc-9a81-747f02606c24 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:19.023588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f0b05-ebba-44fc-9a81-747f02606c24 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:19.024928 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 600/1211] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:39:19.030676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f55e6-ec65-454d-ba19-da09f98cb2eb None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:19.031117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f55e6-ec65-454d-ba19-da09f98cb2eb None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:19.031221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f55e6-ec65-454d-ba19-da09f98cb2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:19.038876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=81 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:39:19.052439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-af6f55e6-ec65-454d-ba19-da09f98cb2eb None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:39:19.053636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-af6f55e6-ec65-454d-ba19-da09f98cb2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:39:19.061593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 612/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:39:19.365431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f0b05-ebba-44fc-9a81-747f02606c24 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:39:19.367075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-767f0b05-ebba-44fc-9a81-747f02606c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:39:19.429749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:39:19.443474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f0b05-ebba-44fc-9a81-747f02606c24 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:39:19.447176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431f9e8-0bcf-465d-aef4-d8b50d261e5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:39:19.447786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f9e8-0bcf-465d-aef4-d8b50d261e5a service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:19.448070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f9e8-0bcf-465d-aef4-d8b50d261e5a service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:19.448370 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f9e8-0bcf-465d-aef4-d8b50d261e5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:19.457809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431f9e8-0bcf-465d-aef4-d8b50d261e5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:39:19.459253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431f9e8-0bcf-465d-aef4-d8b50d261e5a service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:39:19.469207 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 601/1213] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:39:21.631864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a80f0-779d-48f7-9c3a-712bf0edf440 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:21.632429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a80f0-779d-48f7-9c3a-712bf0edf440 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:21.632429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a80f0-779d-48f7-9c3a-712bf0edf440 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:21.633717 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 613/1214] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:39:21.639678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eeea8f-df87-46c9-8098-fd3ba5ad2ac4 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:21.640096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eeea8f-df87-46c9-8098-fd3ba5ad2ac4 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:21.640233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eeea8f-df87-46c9-8098-fd3ba5ad2ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:21.659794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-20eeea8f-df87-46c9-8098-fd3ba5ad2ac4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:39:21.661011 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-20eeea8f-df87-46c9-8098-fd3ba5ad2ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:39:21.668736 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 602/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 14:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:39:21.968189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a80f0-779d-48f7-9c3a-712bf0edf440 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:39:21.969878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-362a80f0-779d-48f7-9c3a-712bf0edf440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:39:21.982884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a80f0-779d-48f7-9c3a-712bf0edf440 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:39:21.986375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98be51-2241-42f5-bb67-394a7b0a1f56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:39:21.987225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98be51-2241-42f5-bb67-394a7b0a1f56 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:39:21.987499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98be51-2241-42f5-bb67-394a7b0a1f56 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:39:21.987768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98be51-2241-42f5-bb67-394a7b0a1f56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:39:21.997136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98be51-2241-42f5-bb67-394a7b0a1f56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:39:21.998626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98be51-2241-42f5-bb67-394a7b0a1f56 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:39:22.007688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 614/1216] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:39:31.655128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=11 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:39:32.003590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=8 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:45.366647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fa688-3c3b-445c-9a3a-d2ebd4ec1992 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.367309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fa688-3c3b-445c-9a3a-d2ebd4ec1992 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.367309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fa688-3c3b-445c-9a3a-d2ebd4ec1992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.397909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-a26fa688-3c3b-445c-9a3a-d2ebd4ec1992 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.399190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a26fa688-3c3b-445c-9a3a-d2ebd4ec1992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.413653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 603/1217] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.425438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13aa8f-0283-4ac1-a70a-26cb90392145 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.426056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13aa8f-0283-4ac1-a70a-26cb90392145 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.426056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13aa8f-0283-4ac1-a70a-26cb90392145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.456071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0a13aa8f-0283-4ac1-a70a-26cb90392145 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.457377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0a13aa8f-0283-4ac1-a70a-26cb90392145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.471000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 615/1218] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.475844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fa688-3c3b-445c-9a3a-d2ebd4ec1992 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:45.489195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fa688-3c3b-445c-9a3a-d2ebd4ec1992 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:45.499208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3163-5ba2-4221-b5bd-9d585b6dc13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:45.500022 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3163-5ba2-4221-b5bd-9d585b6dc13e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.500304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3163-5ba2-4221-b5bd-9d585b6dc13e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.500612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3163-5ba2-4221-b5bd-9d585b6dc13e admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.501348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c3163-5ba2-4221-b5bd-9d585b6dc13e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:45.501847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c3163-5ba2-4221-b5bd-9d585b6dc13e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:45.503555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c3163-5ba2-4221-b5bd-9d585b6dc13e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:45.524467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 604/1219] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:45.533481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52f57f-2e50-4422-9935-01653b164a27 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.533827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52f57f-2e50-4422-9935-01653b164a27 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.534038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52f57f-2e50-4422-9935-01653b164a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.539672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3163-5ba2-4221-b5bd-9d585b6dc13e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:45.565993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3163-5ba2-4221-b5bd-9d585b6dc13e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:45.567570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ff52f57f-2e50-4422-9935-01653b164a27 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.571186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ff52f57f-2e50-4422-9935-01653b164a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.577052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c13eb2-30ee-40a7-938b-a08a1b3c53e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:45.578215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13eb2-30ee-40a7-938b-a08a1b3c53e5 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.580118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13eb2-30ee-40a7-938b-a08a1b3c53e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.581231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13eb2-30ee-40a7-938b-a08a1b3c53e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.581934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c13eb2-30ee-40a7-938b-a08a1b3c53e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:45.584282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING py.warnings [None req-f1c13eb2-30ee-40a7-938b-a08a1b3c53e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:40:45.584282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: warnings.warn(msg) Oct 20 14:40:45.584282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: Oct 20 14:40:45.586589 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c13eb2-30ee-40a7-938b-a08a1b3c53e5 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:45.589176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 616/1220] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.599054 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae68b6-1b00-401b-b220-94748dcf0072 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.599362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae68b6-1b00-401b-b220-94748dcf0072 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.599615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae68b6-1b00-401b-b220-94748dcf0072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.639030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-cdae68b6-1b00-401b-b220-94748dcf0072 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.640313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-cdae68b6-1b00-401b-b220-94748dcf0072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.658948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 617/1221] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:40:45.660436 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a7a3b-1e35-458f-8af1-abdf47c75bb4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.660729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a7a3b-1e35-458f-8af1-abdf47c75bb4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.661009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a7a3b-1e35-458f-8af1-abdf47c75bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.674120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 605/1222] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/projects => generated 401 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:45.675386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1778e7-c709-4454-839d-608d845dcd72 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.675673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1778e7-c709-4454-839d-608d845dcd72 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.675953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1778e7-c709-4454-839d-608d845dcd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.714197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-087a7a3b-1e35-458f-8af1-abdf47c75bb4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.716556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-087a7a3b-1e35-458f-8af1-abdf47c75bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.722057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6b1778e7-c709-4454-839d-608d845dcd72 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.723247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6b1778e7-c709-4454-839d-608d845dcd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.809533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 618/1223] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.810246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a7a3b-1e35-458f-8af1-abdf47c75bb4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:45.818981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 606/1224] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.819743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1778e7-c709-4454-839d-608d845dcd72 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:45.822122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a7a3b-1e35-458f-8af1-abdf47c75bb4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:45.828947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05c11c-e344-405a-9db9-737fc4833c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:45.832386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1778e7-c709-4454-839d-608d845dcd72 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:45.834028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05c11c-e344-405a-9db9-737fc4833c7b admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.834914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05c11c-e344-405a-9db9-737fc4833c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.835293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05c11c-e344-405a-9db9-737fc4833c7b admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.836156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05c11c-e344-405a-9db9-737fc4833c7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:45.836532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05c11c-e344-405a-9db9-737fc4833c7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:45.837701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING py.warnings [None req-9c05c11c-e344-405a-9db9-737fc4833c7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:40:45.837701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: warnings.warn(msg) Oct 20 14:40:45.837701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: Oct 20 14:40:45.838604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05c11c-e344-405a-9db9-737fc4833c7b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:45.838733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644a755-88cb-446b-ab87-708e38d0453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:45.839327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644a755-88cb-446b-ab87-708e38d0453e admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.839605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644a755-88cb-446b-ab87-708e38d0453e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.839875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644a755-88cb-446b-ab87-708e38d0453e admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.841375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644a755-88cb-446b-ab87-708e38d0453e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:45.843123 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644a755-88cb-446b-ab87-708e38d0453e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:45.856694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 619/1225] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:45.856694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ad8d6-f772-4020-8f64-4351f5c75a4a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.856694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ad8d6-f772-4020-8f64-4351f5c75a4a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.856694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ad8d6-f772-4020-8f64-4351f5c75a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.866312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 607/1226] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/users => generated 301 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:45.867612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f2778-e993-41b3-bd30-bcdff872b8a8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.867898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f2778-e993-41b3-bd30-bcdff872b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.868172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f2778-e993-41b3-bd30-bcdff872b8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.875355 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-3c1ad8d6-f772-4020-8f64-4351f5c75a4a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.876609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3c1ad8d6-f772-4020-8f64-4351f5c75a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.884725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 620/1227] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.885911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343123cf-f100-42ba-870e-506f601d7899 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.886199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343123cf-f100-42ba-870e-506f601d7899 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.886477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343123cf-f100-42ba-870e-506f601d7899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.888202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-3a5f2778-e993-41b3-bd30-bcdff872b8a8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.889465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3a5f2778-e993-41b3-bd30-bcdff872b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.900684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 608/1228] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.902110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be95c92-e036-4965-8f18-85c1ba834a0d None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.902422 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be95c92-e036-4965-8f18-85c1ba834a0d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.903359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be95c92-e036-4965-8f18-85c1ba834a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.910820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-343123cf-f100-42ba-870e-506f601d7899 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.911999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-343123cf-f100-42ba-870e-506f601d7899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.920596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 621/1229] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.921779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf8401-98df-4f20-92c3-b609926509da None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.922060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf8401-98df-4f20-92c3-b609926509da None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.922330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf8401-98df-4f20-92c3-b609926509da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.930635 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5be95c92-e036-4965-8f18-85c1ba834a0d None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.931841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5be95c92-e036-4965-8f18-85c1ba834a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.940160 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 609/1230] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.941441 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c87fa5-a5bd-4e0c-99bd-ccf1e4eab56c None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.941729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c87fa5-a5bd-4e0c-99bd-ccf1e4eab56c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.942002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c87fa5-a5bd-4e0c-99bd-ccf1e4eab56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.942110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-68cf8401-98df-4f20-92c3-b609926509da None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.943261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-68cf8401-98df-4f20-92c3-b609926509da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.951860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 622/1231] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.952722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf8401-98df-4f20-92c3-b609926509da None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:45.961942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf8401-98df-4f20-92c3-b609926509da None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:45.962104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b8c87fa5-a5bd-4e0c-99bd-ccf1e4eab56c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:45.963218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b8c87fa5-a5bd-4e0c-99bd-ccf1e4eab56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:45.967294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984fc33-628f-4a7f-b883-d7c19ccccb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:45.967883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984fc33-628f-4a7f-b883-d7c19ccccb99 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.968171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984fc33-628f-4a7f-b883-d7c19ccccb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.968474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984fc33-628f-4a7f-b883-d7c19ccccb99 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.969103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984fc33-628f-4a7f-b883-d7c19ccccb99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:45.970999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984fc33-628f-4a7f-b883-d7c19ccccb99 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:45.971758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 610/1232] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:45.976492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c87fa5-a5bd-4e0c-99bd-ccf1e4eab56c None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:45.982177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c87fa5-a5bd-4e0c-99bd-ccf1e4eab56c None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:45.987305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ede4d-0ed3-407d-a869-d830c8d5c9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:45.988056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ede4d-0ed3-407d-a869-d830c8d5c9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:45.988201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ede4d-0ed3-407d-a869-d830c8d5c9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:45.988626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ede4d-0ed3-407d-a869-d830c8d5c9c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:45.989297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ede4d-0ed3-407d-a869-d830c8d5c9c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:45.991119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ede4d-0ed3-407d-a869-d830c8d5c9c2 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:45.999939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 623/1233] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:45 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.004576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 611/1234] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:46.004737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eb352-0fc9-46a8-8ff6-3c40e7182dfa None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.004846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156cd07-66d3-42d1-af40-4a2568116772 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.004939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eb352-0fc9-46a8-8ff6-3c40e7182dfa None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.005044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eb352-0fc9-46a8-8ff6-3c40e7182dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.005132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156cd07-66d3-42d1-af40-4a2568116772 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.005342 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156cd07-66d3-42d1-af40-4a2568116772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.025155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-504eb352-0fc9-46a8-8ff6-3c40e7182dfa None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:46.026329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5156cd07-66d3-42d1-af40-4a2568116772 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:46.026447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-504eb352-0fc9-46a8-8ff6-3c40e7182dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:46.027499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5156cd07-66d3-42d1-af40-4a2568116772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:46.086225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 624/1235] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:46.087474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dcb88-cbac-4008-b4a0-d57e9eb25ff4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.087762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dcb88-cbac-4008-b4a0-d57e9eb25ff4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.088031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dcb88-cbac-4008-b4a0-d57e9eb25ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.090286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 612/1236] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:46.091365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156cd07-66d3-42d1-af40-4a2568116772 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.096498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:46.103013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156cd07-66d3-42d1-af40-4a2568116772 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.109180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b829a5b-279c-4971-88d4-027640f15a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.109855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b829a5b-279c-4971-88d4-027640f15a09 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.110216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b829a5b-279c-4971-88d4-027640f15a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.110492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b829a5b-279c-4971-88d4-027640f15a09 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.110787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-fb3dcb88-cbac-4008-b4a0-d57e9eb25ff4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:46.111384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b829a5b-279c-4971-88d4-027640f15a09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:46.111687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b829a5b-279c-4971-88d4-027640f15a09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.112234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fb3dcb88-cbac-4008-b4a0-d57e9eb25ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:46.113320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b829a5b-279c-4971-88d4-027640f15a09 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.120091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 625/1237] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:46.120911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dcb88-cbac-4008-b4a0-d57e9eb25ff4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.127578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 613/1238] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:46.128889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1f465-085c-4936-b702-e91351e57dfb None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.129175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1f465-085c-4936-b702-e91351e57dfb None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.129448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1f465-085c-4936-b702-e91351e57dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.130708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dcb88-cbac-4008-b4a0-d57e9eb25ff4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.135333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b03c2-dc71-4bee-af56-6a9961e51b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.135930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b03c2-dc71-4bee-af56-6a9961e51b70 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.136212 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b03c2-dc71-4bee-af56-6a9961e51b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.136519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b03c2-dc71-4bee-af56-6a9961e51b70 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.137198 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b03c2-dc71-4bee-af56-6a9961e51b70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:46.137502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b03c2-dc71-4bee-af56-6a9961e51b70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.139331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b03c2-dc71-4bee-af56-6a9961e51b70 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.148631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-c1e1f465-085c-4936-b702-e91351e57dfb None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:46.149760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c1e1f465-085c-4936-b702-e91351e57dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:46.150940 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 626/1239] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:46.151709 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b03c2-dc71-4bee-af56-6a9961e51b70 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.161422 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 614/1240] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:46.165663 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1f465-085c-4936-b702-e91351e57dfb None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.166547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b03c2-dc71-4bee-af56-6a9961e51b70 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.171738 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33e68d-e8d1-4610-b8e9-addedd689e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.172362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33e68d-e8d1-4610-b8e9-addedd689e72 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.172645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33e68d-e8d1-4610-b8e9-addedd689e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.172935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33e68d-e8d1-4610-b8e9-addedd689e72 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.173580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33e68d-e8d1-4610-b8e9-addedd689e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.174596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING py.warnings [None req-9a33e68d-e8d1-4610-b8e9-addedd689e72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:40:46.174596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: warnings.warn(msg) Oct 20 14:40:46.174596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: Oct 20 14:40:46.175737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1f465-085c-4936-b702-e91351e57dfb None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.175862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33e68d-e8d1-4610-b8e9-addedd689e72 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.180265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc479-7998-450c-8863-f031c2930939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.180883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc479-7998-450c-8863-f031c2930939 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.181165 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc479-7998-450c-8863-f031c2930939 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.181440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc479-7998-450c-8863-f031c2930939 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.182080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fc479-7998-450c-8863-f031c2930939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.183665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fc479-7998-450c-8863-f031c2930939 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.184140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:46.194134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 615/1241] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:46.197469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc479-7998-450c-8863-f031c2930939 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.201678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 627/1242] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.202492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33e68d-e8d1-4610-b8e9-addedd689e72 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.207973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc479-7998-450c-8863-f031c2930939 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.212706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdd09b-a337-4cf3-bac4-9ab074a020a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.213297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdd09b-a337-4cf3-bac4-9ab074a020a9 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.213442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33e68d-e8d1-4610-b8e9-addedd689e72 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.213584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdd09b-a337-4cf3-bac4-9ab074a020a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.213858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdd09b-a337-4cf3-bac4-9ab074a020a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.214496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdd09b-a337-4cf3-bac4-9ab074a020a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:46.214869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdd09b-a337-4cf3-bac4-9ab074a020a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.217372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdd09b-a337-4cf3-bac4-9ab074a020a9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.218598 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b420b5-3ad0-43e0-9474-2c1824cd4125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.219185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b420b5-3ad0-43e0-9474-2c1824cd4125 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.219464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b420b5-3ad0-43e0-9474-2c1824cd4125 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.219737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b420b5-3ad0-43e0-9474-2c1824cd4125 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.220455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b420b5-3ad0-43e0-9474-2c1824cd4125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:46.220761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b420b5-3ad0-43e0-9474-2c1824cd4125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.222337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b420b5-3ad0-43e0-9474-2c1824cd4125 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.229466 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 616/1243] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:46.230269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdd09b-a337-4cf3-bac4-9ab074a020a9 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.236239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 628/1244] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:46.236239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b420b5-3ad0-43e0-9474-2c1824cd4125 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.245747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b420b5-3ad0-43e0-9474-2c1824cd4125 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.247077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdd09b-a337-4cf3-bac4-9ab074a020a9 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.250461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2293d6b-c30e-4343-8708-f54b3917c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.251054 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2293d6b-c30e-4343-8708-f54b3917c42d admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.251334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2293d6b-c30e-4343-8708-f54b3917c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.251608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2293d6b-c30e-4343-8708-f54b3917c42d admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.251840 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddcd7a-94ea-4572-8bf4-e807ac0b87e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.252238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2293d6b-c30e-4343-8708-f54b3917c42d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.252473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddcd7a-94ea-4572-8bf4-e807ac0b87e4 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.252753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddcd7a-94ea-4572-8bf4-e807ac0b87e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.253036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddcd7a-94ea-4572-8bf4-e807ac0b87e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.253677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddcd7a-94ea-4572-8bf4-e807ac0b87e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:46.253998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddcd7a-94ea-4572-8bf4-e807ac0b87e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.254176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2293d6b-c30e-4343-8708-f54b3917c42d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.255575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddcd7a-94ea-4572-8bf4-e807ac0b87e4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.268950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 617/1245] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:46.269809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddcd7a-94ea-4572-8bf4-e807ac0b87e4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.282053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddcd7a-94ea-4572-8bf4-e807ac0b87e4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.282757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 629/1246] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.283618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2293d6b-c30e-4343-8708-f54b3917c42d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.342729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d6043-2cd1-4436-a9b3-4d101abd78d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.343641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d6043-2cd1-4436-a9b3-4d101abd78d0 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.343957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d6043-2cd1-4436-a9b3-4d101abd78d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.344233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d6043-2cd1-4436-a9b3-4d101abd78d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.344914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d6043-2cd1-4436-a9b3-4d101abd78d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.345511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2293d6b-c30e-4343-8708-f54b3917c42d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.346842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d6043-2cd1-4436-a9b3-4d101abd78d0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.350267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dec10-08fd-441d-a75e-27d373a9261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.350904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dec10-08fd-441d-a75e-27d373a9261e admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.351281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dec10-08fd-441d-a75e-27d373a9261e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.351479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dec10-08fd-441d-a75e-27d373a9261e admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.352138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dec10-08fd-441d-a75e-27d373a9261e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:46.352463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dec10-08fd-441d-a75e-27d373a9261e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.354062 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dec10-08fd-441d-a75e-27d373a9261e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.366466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 630/1247] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:46.367271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dec10-08fd-441d-a75e-27d373a9261e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.378147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dec10-08fd-441d-a75e-27d373a9261e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.382860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df67de1-f9bf-49c5-a3aa-0046893c5bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.383504 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67de1-f9bf-49c5-a3aa-0046893c5bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.383781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67de1-f9bf-49c5-a3aa-0046893c5bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.384055 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67de1-f9bf-49c5-a3aa-0046893c5bc9 admin admin] PATH_INFO: `/v3/projects/cee0a6ee88fa492ab67c8913c5b71117/users/d02fe89eaf6d4066972909e293b66388/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.385104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 618/1248] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.385889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d6043-2cd1-4436-a9b3-4d101abd78d0 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.399649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df67de1-f9bf-49c5-a3aa-0046893c5bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee0a6ee88fa492ab67c8913c5b71117, user_id=d02fe89eaf6d4066972909e293b66388, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.402370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df67de1-f9bf-49c5-a3aa-0046893c5bc9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.415543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 631/1249] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:46 2022] PUT /identity/v3/projects/cee0a6ee88fa492ab67c8913c5b71117/users/d02fe89eaf6d4066972909e293b66388/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:46.416601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df67de1-f9bf-49c5-a3aa-0046893c5bc9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.451083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d6043-2cd1-4436-a9b3-4d101abd78d0 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.484027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:46.484763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df67de1-f9bf-49c5-a3aa-0046893c5bc9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.490282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03bf51-af3f-4a3f-82fe-6fb40c279c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.490897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03bf51-af3f-4a3f-82fe-6fb40c279c4d admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.491184 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03bf51-af3f-4a3f-82fe-6fb40c279c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.491497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03bf51-af3f-4a3f-82fe-6fb40c279c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.492132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03bf51-af3f-4a3f-82fe-6fb40c279c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.494086 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03bf51-af3f-4a3f-82fe-6fb40c279c4d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.510082 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb020ba3-0d82-4861-a8bb-c9cc77b6b4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.510771 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb020ba3-0d82-4861-a8bb-c9cc77b6b4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.511052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb020ba3-0d82-4861-a8bb-c9cc77b6b4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.511376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb020ba3-0d82-4861-a8bb-c9cc77b6b4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.512648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb020ba3-0d82-4861-a8bb-c9cc77b6b4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.514679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb020ba3-0d82-4861-a8bb-c9cc77b6b4d9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.522771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 632/1250] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.523570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03bf51-af3f-4a3f-82fe-6fb40c279c4d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.524636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 619/1251] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:46.525431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb020ba3-0d82-4861-a8bb-c9cc77b6b4d9 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.537120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:46.539193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb020ba3-0d82-4861-a8bb-c9cc77b6b4d9 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.592682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03bf51-af3f-4a3f-82fe-6fb40c279c4d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.597445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0564fc-2033-4525-ad68-ea7b7ba36133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.597617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ea58b-c764-404b-a7a0-1906ddda47ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.598058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0564fc-2033-4525-ad68-ea7b7ba36133 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.598198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ea58b-c764-404b-a7a0-1906ddda47ff admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.598342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0564fc-2033-4525-ad68-ea7b7ba36133 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.598485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ea58b-c764-404b-a7a0-1906ddda47ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.598625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0564fc-2033-4525-ad68-ea7b7ba36133 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.598726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ea58b-c764-404b-a7a0-1906ddda47ff admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.599264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0564fc-2033-4525-ad68-ea7b7ba36133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.599387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ea58b-c764-404b-a7a0-1906ddda47ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.601258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0564fc-2033-4525-ad68-ea7b7ba36133 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.601386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ea58b-c764-404b-a7a0-1906ddda47ff admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.628646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 620/1252] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.629302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 633/1253] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.629905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0564fc-2033-4525-ad68-ea7b7ba36133 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.632494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ea58b-c764-404b-a7a0-1906ddda47ff admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.692512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0564fc-2033-4525-ad68-ea7b7ba36133 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.692719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ea58b-c764-404b-a7a0-1906ddda47ff admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.697275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d9dff-a9f9-4b4d-8f11-89c39ab1007b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.697486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814c01e-de6d-4f26-aed1-dbfa08d9bd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.697890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d9dff-a9f9-4b4d-8f11-89c39ab1007b admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.698084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814c01e-de6d-4f26-aed1-dbfa08d9bd72 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.698190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d9dff-a9f9-4b4d-8f11-89c39ab1007b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.698366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814c01e-de6d-4f26-aed1-dbfa08d9bd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.698469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d9dff-a9f9-4b4d-8f11-89c39ab1007b admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.698636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814c01e-de6d-4f26-aed1-dbfa08d9bd72 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.699049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d9dff-a9f9-4b4d-8f11-89c39ab1007b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.699253 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814c01e-de6d-4f26-aed1-dbfa08d9bd72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.701253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d9dff-a9f9-4b4d-8f11-89c39ab1007b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.701363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814c01e-de6d-4f26-aed1-dbfa08d9bd72 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.722617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 621/1254] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.723416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d9dff-a9f9-4b4d-8f11-89c39ab1007b admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.727740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 634/1255] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.729796 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4f6bb-900e-429a-83b3-57542a28a46f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.730136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4f6bb-900e-429a-83b3-57542a28a46f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.730411 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4f6bb-900e-429a-83b3-57542a28a46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.733632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d9dff-a9f9-4b4d-8f11-89c39ab1007b admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.760282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4ce4f6bb-900e-429a-83b3-57542a28a46f None None] MFA Rules not processed for user `d02fe89eaf6d4066972909e293b66388`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:46.761522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4ce4f6bb-900e-429a-83b3-57542a28a46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:46.778745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:46.791617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569dd2c-8f2a-4fe6-9b4c-7fb80cbeee97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.792222 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569dd2c-8f2a-4fe6-9b4c-7fb80cbeee97 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.792529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569dd2c-8f2a-4fe6-9b4c-7fb80cbeee97 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.792826 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569dd2c-8f2a-4fe6-9b4c-7fb80cbeee97 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.794095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569dd2c-8f2a-4fe6-9b4c-7fb80cbeee97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.796038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569dd2c-8f2a-4fe6-9b4c-7fb80cbeee97 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.816632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 622/1256] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.817655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569dd2c-8f2a-4fe6-9b4c-7fb80cbeee97 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.827591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569dd2c-8f2a-4fe6-9b4c-7fb80cbeee97 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.829418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 635/1257] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:46.832844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4f6bb-900e-429a-83b3-57542a28a46f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.832938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f90d4-f7a0-463d-b9e2-07db76d4eb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.832938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f90d4-f7a0-463d-b9e2-07db76d4eb56 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.833120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f90d4-f7a0-463d-b9e2-07db76d4eb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.833335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f90d4-f7a0-463d-b9e2-07db76d4eb56 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.833959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f90d4-f7a0-463d-b9e2-07db76d4eb56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.835879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f90d4-f7a0-463d-b9e2-07db76d4eb56 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.841249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4f6bb-900e-429a-83b3-57542a28a46f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.847183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76980dab-8165-4cc8-a56a-686e77594979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.847867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76980dab-8165-4cc8-a56a-686e77594979 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.848166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76980dab-8165-4cc8-a56a-686e77594979 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.848489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76980dab-8165-4cc8-a56a-686e77594979 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.849190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76980dab-8165-4cc8-a56a-686e77594979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.850885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76980dab-8165-4cc8-a56a-686e77594979 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.853069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:46.860213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 623/1258] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.860213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f90d4-f7a0-463d-b9e2-07db76d4eb56 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.862925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 636/1259] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:46.865779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76980dab-8165-4cc8-a56a-686e77594979 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.868420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f90d4-f7a0-463d-b9e2-07db76d4eb56 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.873695 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89bc1f-b76c-4568-a99f-616244525367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.874293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89bc1f-b76c-4568-a99f-616244525367 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.874436 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76980dab-8165-4cc8-a56a-686e77594979 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.874607 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89bc1f-b76c-4568-a99f-616244525367 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.874900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89bc1f-b76c-4568-a99f-616244525367 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.875526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89bc1f-b76c-4568-a99f-616244525367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.877465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89bc1f-b76c-4568-a99f-616244525367 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.879475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95e993-4e41-4f88-a21d-852fca309d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.880064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95e993-4e41-4f88-a21d-852fca309d67 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.880382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95e993-4e41-4f88-a21d-852fca309d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.880646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95e993-4e41-4f88-a21d-852fca309d67 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.881270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95e993-4e41-4f88-a21d-852fca309d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.883159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95e993-4e41-4f88-a21d-852fca309d67 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.898331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 624/1260] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/users => generated 283 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.899255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89bc1f-b76c-4568-a99f-616244525367 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.910526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 637/1261] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.910526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95e993-4e41-4f88-a21d-852fca309d67 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.910808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89bc1f-b76c-4568-a99f-616244525367 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.913912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4b995-a8dc-4689-9ae3-a7a23e888950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.914503 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b995-a8dc-4689-9ae3-a7a23e888950 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.914778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b995-a8dc-4689-9ae3-a7a23e888950 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.915047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b995-a8dc-4689-9ae3-a7a23e888950 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.915676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4b995-a8dc-4689-9ae3-a7a23e888950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.917319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4b995-a8dc-4689-9ae3-a7a23e888950 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.917988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95e993-4e41-4f88-a21d-852fca309d67 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.923236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7b1e2-e5aa-47fb-a180-7d2f1019aa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.923822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7b1e2-e5aa-47fb-a180-7d2f1019aa96 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.924096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7b1e2-e5aa-47fb-a180-7d2f1019aa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.924395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7b1e2-e5aa-47fb-a180-7d2f1019aa96 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.925021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7b1e2-e5aa-47fb-a180-7d2f1019aa96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.926892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7b1e2-e5aa-47fb-a180-7d2f1019aa96 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.928301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 625/1262] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:46.928301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4b995-a8dc-4689-9ae3-a7a23e888950 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.939289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4b995-a8dc-4689-9ae3-a7a23e888950 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.944835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a9cca-b23d-4ae9-94d6-0edbf33ee2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.945376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a9cca-b23d-4ae9-94d6-0edbf33ee2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.945687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a9cca-b23d-4ae9-94d6-0edbf33ee2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.945943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a9cca-b23d-4ae9-94d6-0edbf33ee2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.946585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a9cca-b23d-4ae9-94d6-0edbf33ee2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.948225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a9cca-b23d-4ae9-94d6-0edbf33ee2d6 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.948951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 638/1263] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:46.950158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7b1e2-e5aa-47fb-a180-7d2f1019aa96 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.959434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7b1e2-e5aa-47fb-a180-7d2f1019aa96 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.961397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 626/1264] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:46.962279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a9cca-b23d-4ae9-94d6-0edbf33ee2d6 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:46.967196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048a8d5-9b99-461d-880b-4a55dc12e331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.967772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048a8d5-9b99-461d-880b-4a55dc12e331 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.968047 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048a8d5-9b99-461d-880b-4a55dc12e331 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.968318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048a8d5-9b99-461d-880b-4a55dc12e331 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.968977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048a8d5-9b99-461d-880b-4a55dc12e331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.970893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048a8d5-9b99-461d-880b-4a55dc12e331 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.972007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a9cca-b23d-4ae9-94d6-0edbf33ee2d6 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:46.976850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc6710-4627-42d8-91de-642ca0e2a540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:46.977471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc6710-4627-42d8-91de-642ca0e2a540 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:46.977745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc6710-4627-42d8-91de-642ca0e2a540 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:46.978017 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc6710-4627-42d8-91de-642ca0e2a540 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:46.978675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc6710-4627-42d8-91de-642ca0e2a540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:46.980256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc6710-4627-42d8-91de-642ca0e2a540 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:46.992709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 627/1265] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:46.993573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc6710-4627-42d8-91de-642ca0e2a540 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.002575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 639/1266] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:46 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:47.002575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048a8d5-9b99-461d-880b-4a55dc12e331 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.003657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc6710-4627-42d8-91de-642ca0e2a540 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.059885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40c84e-4656-435c-a2e3-293eee9d407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.060572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40c84e-4656-435c-a2e3-293eee9d407d admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.060856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40c84e-4656-435c-a2e3-293eee9d407d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.061133 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40c84e-4656-435c-a2e3-293eee9d407d admin admin] PATH_INFO: `/v3/projects/22dfe6d702cb4ad690238da1a6c20163/users/f14f13f618ac44e9a1f463be2852b368/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.064169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048a8d5-9b99-461d-880b-4a55dc12e331 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.068753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877baeb-d565-4cf8-9a9b-3deacdd9970f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.069350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877baeb-d565-4cf8-9a9b-3deacdd9970f admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.069631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877baeb-d565-4cf8-9a9b-3deacdd9970f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.069899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877baeb-d565-4cf8-9a9b-3deacdd9970f admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.070533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877baeb-d565-4cf8-9a9b-3deacdd9970f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.072160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877baeb-d565-4cf8-9a9b-3deacdd9970f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.076301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40c84e-4656-435c-a2e3-293eee9d407d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dfe6d702cb4ad690238da1a6c20163, user_id=f14f13f618ac44e9a1f463be2852b368, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.079000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40c84e-4656-435c-a2e3-293eee9d407d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.082901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 640/1267] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.083893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877baeb-d565-4cf8-9a9b-3deacdd9970f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.094940 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877baeb-d565-4cf8-9a9b-3deacdd9970f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.095684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 628/1268] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:46 2022] PUT /identity/v3/projects/22dfe6d702cb4ad690238da1a6c20163/users/f14f13f618ac44e9a1f463be2852b368/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:47.104475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40c84e-4656-435c-a2e3-293eee9d407d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.147320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:47.159653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40c84e-4656-435c-a2e3-293eee9d407d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.161243 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5cbed-8a0c-4545-886b-ebab8f1319a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.161853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5cbed-8a0c-4545-886b-ebab8f1319a9 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.162155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5cbed-8a0c-4545-886b-ebab8f1319a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.162426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5cbed-8a0c-4545-886b-ebab8f1319a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.163060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5cbed-8a0c-4545-886b-ebab8f1319a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.165147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5cbed-8a0c-4545-886b-ebab8f1319a9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.165272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af818a-5c5b-44c3-afd1-65550c4d73e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.165902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af818a-5c5b-44c3-afd1-65550c4d73e6 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.166225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af818a-5c5b-44c3-afd1-65550c4d73e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.166503 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af818a-5c5b-44c3-afd1-65550c4d73e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.167142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af818a-5c5b-44c3-afd1-65550c4d73e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.168936 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af818a-5c5b-44c3-afd1-65550c4d73e6 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.175312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 641/1269] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.176272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5cbed-8a0c-4545-886b-ebab8f1319a9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.181994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 629/1270] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.182765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af818a-5c5b-44c3-afd1-65550c4d73e6 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.190349 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5cbed-8a0c-4545-886b-ebab8f1319a9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.192863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af818a-5c5b-44c3-afd1-65550c4d73e6 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.195248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa00ad-f38b-44b6-a84b-b8dc75963348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.195774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa00ad-f38b-44b6-a84b-b8dc75963348 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.196049 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa00ad-f38b-44b6-a84b-b8dc75963348 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.196318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa00ad-f38b-44b6-a84b-b8dc75963348 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.197092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa00ad-f38b-44b6-a84b-b8dc75963348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.197469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc29c2-9d2e-4f1d-a123-220f90f8066a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.198051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc29c2-9d2e-4f1d-a123-220f90f8066a admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.198334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc29c2-9d2e-4f1d-a123-220f90f8066a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.198612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc29c2-9d2e-4f1d-a123-220f90f8066a admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.198715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa00ad-f38b-44b6-a84b-b8dc75963348 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.199240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc29c2-9d2e-4f1d-a123-220f90f8066a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.201585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc29c2-9d2e-4f1d-a123-220f90f8066a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.209995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 642/1271] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.210789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa00ad-f38b-44b6-a84b-b8dc75963348 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.214311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 630/1272] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.214311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc29c2-9d2e-4f1d-a123-220f90f8066a admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.220892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa00ad-f38b-44b6-a84b-b8dc75963348 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.223500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc29c2-9d2e-4f1d-a123-220f90f8066a admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.225388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e06e01-ad0d-427d-8274-1480be4b93a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.225960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e06e01-ad0d-427d-8274-1480be4b93a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.226235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e06e01-ad0d-427d-8274-1480be4b93a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.226502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e06e01-ad0d-427d-8274-1480be4b93a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.227120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e06e01-ad0d-427d-8274-1480be4b93a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.228166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71121832-bca4-4254-ba41-9f81c29c2d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.228738 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e06e01-ad0d-427d-8274-1480be4b93a1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.228855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71121832-bca4-4254-ba41-9f81c29c2d4c admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.229054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71121832-bca4-4254-ba41-9f81c29c2d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.229331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71121832-bca4-4254-ba41-9f81c29c2d4c admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.229957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71121832-bca4-4254-ba41-9f81c29c2d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.231891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71121832-bca4-4254-ba41-9f81c29c2d4c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.242078 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 643/1273] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.242078 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e06e01-ad0d-427d-8274-1480be4b93a1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.253188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e06e01-ad0d-427d-8274-1480be4b93a1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.258879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 631/1274] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:47 2022] POST /identity/v3/users => generated 309 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:47.258879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db97ac-c12a-4867-aec3-6e0652e97d05 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.258879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db97ac-c12a-4867-aec3-6e0652e97d05 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.258879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db97ac-c12a-4867-aec3-6e0652e97d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.260136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3028d5-6946-46ee-a22b-ee1f257f1808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.260779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3028d5-6946-46ee-a22b-ee1f257f1808 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.261056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3028d5-6946-46ee-a22b-ee1f257f1808 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.261450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3028d5-6946-46ee-a22b-ee1f257f1808 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.262230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3028d5-6946-46ee-a22b-ee1f257f1808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.263794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3028d5-6946-46ee-a22b-ee1f257f1808 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.269222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:47.275473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 644/1275] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.276518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3028d5-6946-46ee-a22b-ee1f257f1808 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.287655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b3db97ac-c12a-4867-aec3-6e0652e97d05 None None] MFA Rules not processed for user `f14f13f618ac44e9a1f463be2852b368`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:47.288559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3028d5-6946-46ee-a22b-ee1f257f1808 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.288934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b3db97ac-c12a-4867-aec3-6e0652e97d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:47.293976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee167e3-9ea4-4043-bc44-d432a40acf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.295055 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee167e3-9ea4-4043-bc44-d432a40acf98 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.295341 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee167e3-9ea4-4043-bc44-d432a40acf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.295623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee167e3-9ea4-4043-bc44-d432a40acf98 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.296255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee167e3-9ea4-4043-bc44-d432a40acf98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.297877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee167e3-9ea4-4043-bc44-d432a40acf98 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.308005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 645/1276] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.308958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee167e3-9ea4-4043-bc44-d432a40acf98 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.318484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee167e3-9ea4-4043-bc44-d432a40acf98 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.323067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74394288-4cbc-4d76-be5b-2d693b09f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.323697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74394288-4cbc-4d76-be5b-2d693b09f1d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.323982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74394288-4cbc-4d76-be5b-2d693b09f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.324260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74394288-4cbc-4d76-be5b-2d693b09f1d8 admin admin] PATH_INFO: `/v3/projects/1ebb3ea35947435b827833547073f0a5/users/acc03422a05d4fc0ad680bf5aeb58d5c/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.339050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74394288-4cbc-4d76-be5b-2d693b09f1d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ebb3ea35947435b827833547073f0a5, user_id=acc03422a05d4fc0ad680bf5aeb58d5c, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.342387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74394288-4cbc-4d76-be5b-2d693b09f1d8 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.356601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 632/1277] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:47.357434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db97ac-c12a-4867-aec3-6e0652e97d05 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.358294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 646/1278] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:47 2022] PUT /identity/v3/projects/1ebb3ea35947435b827833547073f0a5/users/acc03422a05d4fc0ad680bf5aeb58d5c/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:47.358931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74394288-4cbc-4d76-be5b-2d693b09f1d8 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.372101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db97ac-c12a-4867-aec3-6e0652e97d05 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.431154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74394288-4cbc-4d76-be5b-2d693b09f1d8 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.432962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50567c2a-9233-4f68-b75f-47726aa29e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.434240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50567c2a-9233-4f68-b75f-47726aa29e04 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.434531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50567c2a-9233-4f68-b75f-47726aa29e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.434815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50567c2a-9233-4f68-b75f-47726aa29e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.435524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50567c2a-9233-4f68-b75f-47726aa29e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.436788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571ae7-f0b2-4de2-a6e0-34e1ef340c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.437164 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50567c2a-9233-4f68-b75f-47726aa29e04 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.437583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571ae7-f0b2-4de2-a6e0-34e1ef340c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.437915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571ae7-f0b2-4de2-a6e0-34e1ef340c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.438306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571ae7-f0b2-4de2-a6e0-34e1ef340c2c admin admin] PATH_INFO: `/v3/projects/201aed13c0e342f5b30286ce3f82eb82/users/812daba513084a55badecfcde843a6ff/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.450369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 633/1279] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.450369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50567c2a-9233-4f68-b75f-47726aa29e04 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.453384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59571ae7-f0b2-4de2-a6e0-34e1ef340c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=201aed13c0e342f5b30286ce3f82eb82, user_id=812daba513084a55badecfcde843a6ff, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.456000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59571ae7-f0b2-4de2-a6e0-34e1ef340c2c admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.457295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50567c2a-9233-4f68-b75f-47726aa29e04 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.462890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a45cc-e15c-4558-a3cd-07277594a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.463871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a45cc-e15c-4558-a3cd-07277594a221 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.464093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a45cc-e15c-4558-a3cd-07277594a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.464433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a45cc-e15c-4558-a3cd-07277594a221 admin admin] PATH_INFO: `/v3/projects/8d8d1dfd3c714794a3fca61b2c6b093e/users/e0d4be0e0f744242a323ac9694aef9ad/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.469490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 647/1280] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:47 2022] PUT /identity/v3/projects/201aed13c0e342f5b30286ce3f82eb82/users/812daba513084a55badecfcde843a6ff/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:47.470303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571ae7-f0b2-4de2-a6e0-34e1ef340c2c admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.480256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a45cc-e15c-4558-a3cd-07277594a221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d8d1dfd3c714794a3fca61b2c6b093e, user_id=e0d4be0e0f744242a323ac9694aef9ad, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.483489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a45cc-e15c-4558-a3cd-07277594a221 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.496224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 634/1281] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:47 2022] PUT /identity/v3/projects/8d8d1dfd3c714794a3fca61b2c6b093e/users/e0d4be0e0f744242a323ac9694aef9ad/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:47.500285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a45cc-e15c-4558-a3cd-07277594a221 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.522689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:47.534805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571ae7-f0b2-4de2-a6e0-34e1ef340c2c admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.562038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a45cc-e15c-4558-a3cd-07277594a221 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.566961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b85e1d-6c96-4671-8f8e-7c109d80a370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.567622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b85e1d-6c96-4671-8f8e-7c109d80a370 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.567939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b85e1d-6c96-4671-8f8e-7c109d80a370 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.568220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b85e1d-6c96-4671-8f8e-7c109d80a370 admin admin] PATH_INFO: `/v3/projects/e5e217c805ce4bbc8d0d3cf8ae8e4568/users/6b5c1dbc28bc4c009f2cc2c45dcea2d9/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.582698 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b85e1d-6c96-4671-8f8e-7c109d80a370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e217c805ce4bbc8d0d3cf8ae8e4568, user_id=6b5c1dbc28bc4c009f2cc2c45dcea2d9, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.584988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:47.585687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b85e1d-6c96-4671-8f8e-7c109d80a370 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.593974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc33182-ecff-43b0-820e-bc884fc9dd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.594748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc33182-ecff-43b0-820e-bc884fc9dd04 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.595072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc33182-ecff-43b0-820e-bc884fc9dd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.595660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc33182-ecff-43b0-820e-bc884fc9dd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.596462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc33182-ecff-43b0-820e-bc884fc9dd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.598054 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc33182-ecff-43b0-820e-bc884fc9dd04 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.598676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 635/1282] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:47 2022] PUT /identity/v3/projects/e5e217c805ce4bbc8d0d3cf8ae8e4568/users/6b5c1dbc28bc4c009f2cc2c45dcea2d9/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:47.599462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b85e1d-6c96-4671-8f8e-7c109d80a370 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.608629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 648/1283] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:47.610134 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a91a7-d578-43c6-ae43-bddfd8acb1cd None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.610418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a91a7-d578-43c6-ae43-bddfd8acb1cd None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.610693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a91a7-d578-43c6-ae43-bddfd8acb1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.639913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-105a91a7-d578-43c6-ae43-bddfd8acb1cd None None] MFA Rules not processed for user `acc03422a05d4fc0ad680bf5aeb58d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:47.641239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-105a91a7-d578-43c6-ae43-bddfd8acb1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:47.671041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b85e1d-6c96-4671-8f8e-7c109d80a370 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.675763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c48f4b-b947-4d07-a82a-0ab48337f57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.676443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c48f4b-b947-4d07-a82a-0ab48337f57a admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.676728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c48f4b-b947-4d07-a82a-0ab48337f57a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.677003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c48f4b-b947-4d07-a82a-0ab48337f57a admin admin] PATH_INFO: `/v3/projects/a3d7fadfead1456792446f71c465bc0c/users/4d74d7fd62cd464183f9df4ca581833c/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.691394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c48f4b-b947-4d07-a82a-0ab48337f57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d7fadfead1456792446f71c465bc0c, user_id=4d74d7fd62cd464183f9df4ca581833c, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.694090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c48f4b-b947-4d07-a82a-0ab48337f57a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.706084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 649/1284] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:47.707270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 636/1285] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:47 2022] PUT /identity/v3/projects/a3d7fadfead1456792446f71c465bc0c/users/4d74d7fd62cd464183f9df4ca581833c/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:47.707433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503d71c-d232-45e1-a582-7a3533ee6f69 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.707741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503d71c-d232-45e1-a582-7a3533ee6f69 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.708075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503d71c-d232-45e1-a582-7a3533ee6f69 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.709279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c48f4b-b947-4d07-a82a-0ab48337f57a admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.709511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 650/1286] 10.222.0.13 () {54 vars in 961 bytes} [Thu Oct 20 14:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:40:47.711046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2d13-d27b-4fc7-ae23-fd7d42ff8675 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.711520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2d13-d27b-4fc7-ae23-fd7d42ff8675 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.711815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2d13-d27b-4fc7-ae23-fd7d42ff8675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.730136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0e3c2d13-d27b-4fc7-ae23-fd7d42ff8675 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:47.731893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0e3c2d13-d27b-4fc7-ae23-fd7d42ff8675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:47.773213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c48f4b-b947-4d07-a82a-0ab48337f57a admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.778161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df2c38-26c0-4364-b9af-5c0f539b9ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.778854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df2c38-26c0-4364-b9af-5c0f539b9ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.779140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df2c38-26c0-4364-b9af-5c0f539b9ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.779422 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df2c38-26c0-4364-b9af-5c0f539b9ff7 admin admin] PATH_INFO: `/v3/projects/afd56c15beff4515a44799ee7ec083a3/users/9fc72f6965bb4814a5d85bd04769e4cb/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.790684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 651/1287] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:47.794543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830a56-b1ed-4140-908e-571fb7ada84a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.794543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830a56-b1ed-4140-908e-571fb7ada84a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.794543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830a56-b1ed-4140-908e-571fb7ada84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.799298 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df2c38-26c0-4364-b9af-5c0f539b9ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd56c15beff4515a44799ee7ec083a3, user_id=9fc72f6965bb4814a5d85bd04769e4cb, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.802085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df2c38-26c0-4364-b9af-5c0f539b9ff7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.812635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4a830a56-b1ed-4140-908e-571fb7ada84a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:47.813890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4a830a56-b1ed-4140-908e-571fb7ada84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:47.815868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 637/1288] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:47 2022] PUT /identity/v3/projects/afd56c15beff4515a44799ee7ec083a3/users/9fc72f6965bb4814a5d85bd04769e4cb/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:47.817191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d0a53-29e2-4c11-aa3a-c300ab1ece5e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.817541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d0a53-29e2-4c11-aa3a-c300ab1ece5e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.817878 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d0a53-29e2-4c11-aa3a-c300ab1ece5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.837576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:47.846716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-113d0a53-29e2-4c11-aa3a-c300ab1ece5e None None] MFA Rules not processed for user `6b5c1dbc28bc4c009f2cc2c45dcea2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:47.849215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-113d0a53-29e2-4c11-aa3a-c300ab1ece5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:47.877764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 652/1289] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:47.878586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830a56-b1ed-4140-908e-571fb7ada84a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.889622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830a56-b1ed-4140-908e-571fb7ada84a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.894370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eb629-3492-4714-9cfd-db451fcff065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.895130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb629-3492-4714-9cfd-db451fcff065 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.895488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb629-3492-4714-9cfd-db451fcff065 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.895773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb629-3492-4714-9cfd-db451fcff065 admin admin] PATH_INFO: `/v3/projects/150a7512d51b4ad0bf949e0f625d9a10/users/abe8ee85ccbd4736b3a82073517450f4/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.910528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eb629-3492-4714-9cfd-db451fcff065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=150a7512d51b4ad0bf949e0f625d9a10, user_id=abe8ee85ccbd4736b3a82073517450f4, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.913330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eb629-3492-4714-9cfd-db451fcff065 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:47.916851 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 638/1290] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:47.917747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d0a53-29e2-4c11-aa3a-c300ab1ece5e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:47.926149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:47.928932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d0a53-29e2-4c11-aa3a-c300ab1ece5e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:47.932650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 653/1291] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:47 2022] PUT /identity/v3/projects/150a7512d51b4ad0bf949e0f625d9a10/users/abe8ee85ccbd4736b3a82073517450f4/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:47.934882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04353c-8652-458d-bb2b-b3cc98dc44b6 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.935376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04353c-8652-458d-bb2b-b3cc98dc44b6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.936085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04353c-8652-458d-bb2b-b3cc98dc44b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.956015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-8f04353c-8652-458d-bb2b-b3cc98dc44b6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:47.957282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8f04353c-8652-458d-bb2b-b3cc98dc44b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:47.987126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c0e69-a6f1-4243-a89c-4c004a9a9485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:47.987730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c0e69-a6f1-4243-a89c-4c004a9a9485 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:47.988009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c0e69-a6f1-4243-a89c-4c004a9a9485 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:47.988288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c0e69-a6f1-4243-a89c-4c004a9a9485 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:47.988964 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c0e69-a6f1-4243-a89c-4c004a9a9485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:47.990863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c0e69-a6f1-4243-a89c-4c004a9a9485 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.017534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 639/1292] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:47 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:48.018710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 654/1293] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:48.019032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4881acd-d001-49b5-9bd2-b20f0078c0fb None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.019328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4881acd-d001-49b5-9bd2-b20f0078c0fb None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.019447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04353c-8652-458d-bb2b-b3cc98dc44b6 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.019608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4881acd-d001-49b5-9bd2-b20f0078c0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.020835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8f04353c-8652-458d-bb2b-b3cc98dc44b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:48.058718 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-f4881acd-d001-49b5-9bd2-b20f0078c0fb None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:48.059905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f4881acd-d001-49b5-9bd2-b20f0078c0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:48.079056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04353c-8652-458d-bb2b-b3cc98dc44b6 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.128145 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe15b5c-5fb1-4958-8232-b717b6690eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.128409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 640/1294] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Oct 20 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:48.128787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe15b5c-5fb1-4958-8232-b717b6690eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.129174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe15b5c-5fb1-4958-8232-b717b6690eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.129479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe15b5c-5fb1-4958-8232-b717b6690eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.129575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfe0b0-14c1-4935-b188-e4b4108398a2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.129866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfe0b0-14c1-4935-b188-e4b4108398a2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.130203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfe0b0-14c1-4935-b188-e4b4108398a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.139122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:48.153613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-01dfe0b0-14c1-4935-b188-e4b4108398a2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:48.154818 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-01dfe0b0-14c1-4935-b188-e4b4108398a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:48.195449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe15b5c-5fb1-4958-8232-b717b6690eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.196957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe15b5c-5fb1-4958-8232-b717b6690eb8 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.207408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 655/1295] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:48.208149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe15b5c-5fb1-4958-8232-b717b6690eb8 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.213282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:48.218529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 641/1296] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:48.218969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe15b5c-5fb1-4958-8232-b717b6690eb8 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.219275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfe0b0-14c1-4935-b188-e4b4108398a2 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.224652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ddd73-1c8c-4aa0-a7b6-97b2d1e65026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.225331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ddd73-1c8c-4aa0-a7b6-97b2d1e65026 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.225769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ddd73-1c8c-4aa0-a7b6-97b2d1e65026 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.226122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ddd73-1c8c-4aa0-a7b6-97b2d1e65026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.237571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ddd73-1c8c-4aa0-a7b6-97b2d1e65026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.239028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ddd73-1c8c-4aa0-a7b6-97b2d1e65026 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.251281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 656/1297] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:48.252110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ddd73-1c8c-4aa0-a7b6-97b2d1e65026 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.262310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ddd73-1c8c-4aa0-a7b6-97b2d1e65026 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.266991 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f6eae-f753-4f51-8abe-9f06c350cccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.267756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f6eae-f753-4f51-8abe-9f06c350cccc admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.267908 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f6eae-f753-4f51-8abe-9f06c350cccc admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.268111 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f6eae-f753-4f51-8abe-9f06c350cccc admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.268772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f6eae-f753-4f51-8abe-9f06c350cccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.270398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f6eae-f753-4f51-8abe-9f06c350cccc admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.282713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 657/1298] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:48.283528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f6eae-f753-4f51-8abe-9f06c350cccc admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.290104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfe0b0-14c1-4935-b188-e4b4108398a2 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.295294 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efec88f-45c6-4b27-a9fe-ad1aec1bb9e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.295829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f6eae-f753-4f51-8abe-9f06c350cccc admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.295969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efec88f-45c6-4b27-a9fe-ad1aec1bb9e5 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.296218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efec88f-45c6-4b27-a9fe-ad1aec1bb9e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.296526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efec88f-45c6-4b27-a9fe-ad1aec1bb9e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.300855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fb1f4-76f1-4680-8f1a-0b6283e99e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.301448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fb1f4-76f1-4680-8f1a-0b6283e99e09 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.301724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fb1f4-76f1-4680-8f1a-0b6283e99e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.301989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fb1f4-76f1-4680-8f1a-0b6283e99e09 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.302674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fb1f4-76f1-4680-8f1a-0b6283e99e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.304555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fb1f4-76f1-4680-8f1a-0b6283e99e09 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.328695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 658/1299] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:48 2022] POST /identity/v3/users => generated 299 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:48.329830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fb1f4-76f1-4680-8f1a-0b6283e99e09 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.356779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fb1f4-76f1-4680-8f1a-0b6283e99e09 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.360767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2437f-7baa-4737-a689-e69a939347ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.361780 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2437f-7baa-4737-a689-e69a939347ca service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.362129 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2437f-7baa-4737-a689-e69a939347ca service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.362623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2437f-7baa-4737-a689-e69a939347ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.369905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efec88f-45c6-4b27-a9fe-ad1aec1bb9e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.371386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efec88f-45c6-4b27-a9fe-ad1aec1bb9e5 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.375901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2437f-7baa-4737-a689-e69a939347ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.378309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2437f-7baa-4737-a689-e69a939347ca service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.382369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 642/1300] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:48.383232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efec88f-45c6-4b27-a9fe-ad1aec1bb9e5 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.392856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 659/1301] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:48.393397 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2437f-7baa-4737-a689-e69a939347ca service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.402738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efec88f-45c6-4b27-a9fe-ad1aec1bb9e5 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.408454 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2437f-7baa-4737-a689-e69a939347ca service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.415932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f64b83-7156-4150-ba68-dcf56851fe72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.416841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f64b83-7156-4150-ba68-dcf56851fe72 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.417158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f64b83-7156-4150-ba68-dcf56851fe72 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.417786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f64b83-7156-4150-ba68-dcf56851fe72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.419714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528893b-d13f-4dc5-b15f-20e48a93cb36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.420428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528893b-d13f-4dc5-b15f-20e48a93cb36 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.420689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528893b-d13f-4dc5-b15f-20e48a93cb36 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.421046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528893b-d13f-4dc5-b15f-20e48a93cb36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.439439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528893b-d13f-4dc5-b15f-20e48a93cb36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.448386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528893b-d13f-4dc5-b15f-20e48a93cb36 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.460934 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 660/1302] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:48.461690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528893b-d13f-4dc5-b15f-20e48a93cb36 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.473614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528893b-d13f-4dc5-b15f-20e48a93cb36 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.487033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a91eb-87de-4baa-8531-213004f47fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.488847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a91eb-87de-4baa-8531-213004f47fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.489135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a91eb-87de-4baa-8531-213004f47fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.489418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a91eb-87de-4baa-8531-213004f47fc1 admin admin] PATH_INFO: `/v3/projects/150a7512d51b4ad0bf949e0f625d9a10/users/abe8ee85ccbd4736b3a82073517450f4/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.494481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a91eb-87de-4baa-8531-213004f47fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=150a7512d51b4ad0bf949e0f625d9a10, user_id=abe8ee85ccbd4736b3a82073517450f4, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.501253 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a91eb-87de-4baa-8531-213004f47fc1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.509045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f64b83-7156-4150-ba68-dcf56851fe72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.511092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f64b83-7156-4150-ba68-dcf56851fe72 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.516102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:48.524738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 643/1303] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:48.526161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f64b83-7156-4150-ba68-dcf56851fe72 service nova] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.531910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 661/1304] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:48 2022] PUT /identity/v3/projects/150a7512d51b4ad0bf949e0f625d9a10/users/abe8ee85ccbd4736b3a82073517450f4/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:48.545344 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5baf60-e7b0-4b49-8209-04b3dc359ac1 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.545892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5baf60-e7b0-4b49-8209-04b3dc359ac1 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.545981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5baf60-e7b0-4b49-8209-04b3dc359ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.576734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f64b83-7156-4150-ba68-dcf56851fe72 service nova] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.617019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ea5baf60-e7b0-4b49-8209-04b3dc359ac1 None None] MFA Rules not processed for user `abe8ee85ccbd4736b3a82073517450f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:48.619211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ea5baf60-e7b0-4b49-8209-04b3dc359ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:48.676325 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c15b7-8dd7-46a4-af75-eebc63d3b4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.679134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c15b7-8dd7-46a4-af75-eebc63d3b4ac admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.679794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c15b7-8dd7-46a4-af75-eebc63d3b4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.681679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c15b7-8dd7-46a4-af75-eebc63d3b4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.687181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c15b7-8dd7-46a4-af75-eebc63d3b4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.692216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c15b7-8dd7-46a4-af75-eebc63d3b4ac admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.726703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 644/1305] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:48.727682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c15b7-8dd7-46a4-af75-eebc63d3b4ac admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.742253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c15b7-8dd7-46a4-af75-eebc63d3b4ac admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:48.757026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704caa5-86c8-4323-813d-535a50ed4596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:48.759103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704caa5-86c8-4323-813d-535a50ed4596 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:48.763801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704caa5-86c8-4323-813d-535a50ed4596 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:48.763801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704caa5-86c8-4323-813d-535a50ed4596 admin admin] PATH_INFO: `/v3/projects/6498c6e729ee4c8f818a3a18d6d0790c/users/b19ac02af2f34adb8ceb8d348de05173/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:48.797980 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 662/1306] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:40:48.808083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704caa5-86c8-4323-813d-535a50ed4596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6498c6e729ee4c8f818a3a18d6d0790c, user_id=b19ac02af2f34adb8ceb8d348de05173, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:48.812516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704caa5-86c8-4323-813d-535a50ed4596 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:48.814928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5baf60-e7b0-4b49-8209-04b3dc359ac1 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.853243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 645/1307] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:48 2022] PUT /identity/v3/projects/6498c6e729ee4c8f818a3a18d6d0790c/users/b19ac02af2f34adb8ceb8d348de05173/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:48.858506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704caa5-86c8-4323-813d-535a50ed4596 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:48.947455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5baf60-e7b0-4b49-8209-04b3dc359ac1 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:49.016961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:49.023888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704caa5-86c8-4323-813d-535a50ed4596 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:49.037945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa7bdb-ee88-4470-bdd3-dc89d9823dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:49.038848 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa7bdb-ee88-4470-bdd3-dc89d9823dad admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:49.039191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa7bdb-ee88-4470-bdd3-dc89d9823dad admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:49.039487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa7bdb-ee88-4470-bdd3-dc89d9823dad admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:49.040176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa7bdb-ee88-4470-bdd3-dc89d9823dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:49.042975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa7bdb-ee88-4470-bdd3-dc89d9823dad admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:49.062597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 646/1308] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:49.064568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c54a9-9768-42f0-8621-431f2f7dbeea None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:49.064962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c54a9-9768-42f0-8621-431f2f7dbeea None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:49.065278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c54a9-9768-42f0-8621-431f2f7dbeea None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:49.067344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 647/1309] 10.222.0.13 () {54 vars in 912 bytes} [Thu Oct 20 14:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:40:49.073037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c54a9-9768-42f0-8621-431f2f7dbeea None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:49.074107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202d2b2-273d-4443-9aba-6405b9189bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:49.075995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202d2b2-273d-4443-9aba-6405b9189bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:49.076313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202d2b2-273d-4443-9aba-6405b9189bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:49.076713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202d2b2-273d-4443-9aba-6405b9189bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:49.093539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c54a9-9768-42f0-8621-431f2f7dbeea None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:49.099784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe954f3-1f08-4235-9d3d-2526585dc612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:49.100746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe954f3-1f08-4235-9d3d-2526585dc612 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:49.101154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe954f3-1f08-4235-9d3d-2526585dc612 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:49.101308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe954f3-1f08-4235-9d3d-2526585dc612 admin admin] PATH_INFO: `/v3/projects/6498c6e729ee4c8f818a3a18d6d0790c/users/b19ac02af2f34adb8ceb8d348de05173/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:49.103719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe954f3-1f08-4235-9d3d-2526585dc612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6498c6e729ee4c8f818a3a18d6d0790c, user_id=b19ac02af2f34adb8ceb8d348de05173, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:49.109335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe954f3-1f08-4235-9d3d-2526585dc612 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:49.124449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 648/1310] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:49 2022] PUT /identity/v3/projects/6498c6e729ee4c8f818a3a18d6d0790c/users/b19ac02af2f34adb8ceb8d348de05173/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:49.137337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912f212-61be-4450-9746-5c4c02f6e280 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:49.137585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912f212-61be-4450-9746-5c4c02f6e280 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:49.137896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912f212-61be-4450-9746-5c4c02f6e280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:49.163362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202d2b2-273d-4443-9aba-6405b9189bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:49.166703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6912f212-61be-4450-9746-5c4c02f6e280 None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:49.167955 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6912f212-61be-4450-9746-5c4c02f6e280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:49.196721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202d2b2-273d-4443-9aba-6405b9189bd6 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:49.244948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 663/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:49.258301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1df13d-186a-49b3-98ea-997a41d50d6e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:49.260911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1df13d-186a-49b3-98ea-997a41d50d6e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:49.261200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1df13d-186a-49b3-98ea-997a41d50d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:49.298571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:49.373908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 649/1312] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:49.389574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-8c1df13d-186a-49b3-98ea-997a41d50d6e None None] MFA Rules not processed for user `b19ac02af2f34adb8ceb8d348de05173`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:49.390647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912f212-61be-4450-9746-5c4c02f6e280 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:49.391008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8c1df13d-186a-49b3-98ea-997a41d50d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:49.594450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912f212-61be-4450-9746-5c4c02f6e280 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:49.619425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988628-a36a-48b8-bb12-cedff1e85517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:49.619425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988628-a36a-48b8-bb12-cedff1e85517 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:49.619425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988628-a36a-48b8-bb12-cedff1e85517 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:49.619425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988628-a36a-48b8-bb12-cedff1e85517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:49.627608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 664/1313] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:49.653861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d988628-a36a-48b8-bb12-cedff1e85517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:49.655345 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d988628-a36a-48b8-bb12-cedff1e85517 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:49.676841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1df13d-186a-49b3-98ea-997a41d50d6e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:49.688855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 650/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:40:49.706583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1df13d-186a-49b3-98ea-997a41d50d6e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:49.713729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a1da6-ea82-4241-8bae-5f1aea856839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:49.716826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a1da6-ea82-4241-8bae-5f1aea856839 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:49.717446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a1da6-ea82-4241-8bae-5f1aea856839 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:49.718858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a1da6-ea82-4241-8bae-5f1aea856839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:49.740131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a1da6-ea82-4241-8bae-5f1aea856839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:49.742802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a1da6-ea82-4241-8bae-5f1aea856839 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:49.775524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 665/1315] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:40:49.825505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988628-a36a-48b8-bb12-cedff1e85517 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:50.014434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a1da6-ea82-4241-8bae-5f1aea856839 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:50.061929 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988628-a36a-48b8-bb12-cedff1e85517 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:50.076404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:50.077535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:50.077823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:50.078100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] PATH_INFO: `/v3/users/acc03422a05d4fc0ad680bf5aeb58d5c` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:50.080702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc03422a05d4fc0ad680bf5aeb58d5c)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:50.081921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: WARNING py.warnings [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:40:50.081921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: warnings.warn(msg) Oct 20 14:40:50.081921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: Oct 20 14:40:50.083157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:50.115721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:50.140519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a1da6-ea82-4241-8bae-5f1aea856839 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:50.150234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:50.151284 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:50.151669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:50.152012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] PATH_INFO: `/v3/users/d02fe89eaf6d4066972909e293b66388` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:50.158511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02fe89eaf6d4066972909e293b66388)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:50.164792 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:50.217795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc03422a05d4fc0ad680bf5aeb58d5c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.248680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc03422a05d4fc0ad680bf5aeb58d5c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.250927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc03422a05d4fc0ad680bf5aeb58d5c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.252155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc03422a05d4fc0ad680bf5aeb58d5c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.271898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02fe89eaf6d4066972909e293b66388'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.274371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4af8cd9f-8382-4afd-9c42-0a3afbd61603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc03422a05d4fc0ad680bf5aeb58d5c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.300020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02fe89eaf6d4066972909e293b66388'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.302805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02fe89eaf6d4066972909e293b66388'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.303476 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02fe89eaf6d4066972909e293b66388'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.322915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5adbb16e-fa45-4e45-89b5-9db655b2418e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02fe89eaf6d4066972909e293b66388'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:50.342681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 651/1316] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:40:49 2022] DELETE /identity/v3/users/acc03422a05d4fc0ad680bf5aeb58d5c => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:50.344016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fb74d-24fe-4854-8739-94e8ee447ee1 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:50.344166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fb74d-24fe-4854-8739-94e8ee447ee1 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:50.348571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fb74d-24fe-4854-8739-94e8ee447ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:50.355848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 666/1317] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:40:50 2022] DELETE /identity/v3/users/d02fe89eaf6d4066972909e293b66388 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:40:50.362137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b1fdb-00b2-434e-bd42-3dbef969abce None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:50.364613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b1fdb-00b2-434e-bd42-3dbef969abce None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:50.364920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b1fdb-00b2-434e-bd42-3dbef969abce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:50.375168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:50.385684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-287fb74d-24fe-4854-8739-94e8ee447ee1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:50.391049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-287fb74d-24fe-4854-8739-94e8ee447ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:50.413716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-702b1fdb-00b2-434e-bd42-3dbef969abce None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:50.414971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-702b1fdb-00b2-434e-bd42-3dbef969abce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:50.538767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 652/1318] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:50.572462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 667/1319] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:50.622621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fb74d-24fe-4854-8739-94e8ee447ee1 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:50.624910 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-287fb74d-24fe-4854-8739-94e8ee447ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:50.678799 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b1fdb-00b2-434e-bd42-3dbef969abce None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:50.680445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-702b1fdb-00b2-434e-bd42-3dbef969abce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:50.779248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fb74d-24fe-4854-8739-94e8ee447ee1 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:50.799886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa85b9-3fec-43a0-af0f-5dd4e4daf0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:50.800536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa85b9-3fec-43a0-af0f-5dd4e4daf0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:50.800823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa85b9-3fec-43a0-af0f-5dd4e4daf0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:50.801094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa85b9-3fec-43a0-af0f-5dd4e4daf0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:50.833088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b1fdb-00b2-434e-bd42-3dbef969abce None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:50.837646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa85b9-3fec-43a0-af0f-5dd4e4daf0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:50.842454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa85b9-3fec-43a0-af0f-5dd4e4daf0e6 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:50.843502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fbda1-a88c-417e-b6e0-7b4b530642cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:50.845564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fbda1-a88c-417e-b6e0-7b4b530642cd service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:50.847355 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fbda1-a88c-417e-b6e0-7b4b530642cd service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:50.847987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fbda1-a88c-417e-b6e0-7b4b530642cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:50.874520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7fbda1-a88c-417e-b6e0-7b4b530642cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:50.877122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 653/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:50.877992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7fbda1-a88c-417e-b6e0-7b4b530642cd service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:50.908377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 668/1321] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:40:50.942280 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329988b-f679-4bc2-a9f2-b78771b476ba None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:50.942966 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329988b-f679-4bc2-a9f2-b78771b476ba None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:50.943424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329988b-f679-4bc2-a9f2-b78771b476ba None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:50.945292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 654/1322] 10.222.0.13 () {54 vars in 961 bytes} [Thu Oct 20 14:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:40:50.957723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7769293-fe73-4fb0-9dc6-d20aac95e0d9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:50.958314 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7769293-fe73-4fb0-9dc6-d20aac95e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:50.958612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7769293-fe73-4fb0-9dc6-d20aac95e0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:50.994939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f7769293-fe73-4fb0-9dc6-d20aac95e0d9 None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:50.996757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f7769293-fe73-4fb0-9dc6-d20aac95e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:51.025356 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:51.146833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 669/1323] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Oct 20 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:51.152286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb89083-cff9-4ffe-b08b-9661cd7c4aee None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:51.153149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb89083-cff9-4ffe-b08b-9661cd7c4aee None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:51.153458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb89083-cff9-4ffe-b08b-9661cd7c4aee None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:51.154982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 655/1324] 10.222.0.13 () {54 vars in 912 bytes} [Thu Oct 20 14:40:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:40:51.161795 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd64407-3e83-4947-b14f-534ee23b137c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:51.162757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd64407-3e83-4947-b14f-534ee23b137c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:51.163050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd64407-3e83-4947-b14f-534ee23b137c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:51.209444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9dd64407-3e83-4947-b14f-534ee23b137c None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:51.213584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9dd64407-3e83-4947-b14f-534ee23b137c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:51.233817 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 670/1325] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:40:51.248211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb89083-cff9-4ffe-b08b-9661cd7c4aee None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:51.250252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5bb89083-cff9-4ffe-b08b-9661cd7c4aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:51.336897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb89083-cff9-4ffe-b08b-9661cd7c4aee None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:51.345147 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed3bc7-4bca-4375-9556-89264ae4eb28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:51.345994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed3bc7-4bca-4375-9556-89264ae4eb28 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:51.346374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed3bc7-4bca-4375-9556-89264ae4eb28 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:51.346834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed3bc7-4bca-4375-9556-89264ae4eb28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:51.363329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed3bc7-4bca-4375-9556-89264ae4eb28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:51.364978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed3bc7-4bca-4375-9556-89264ae4eb28 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:51.406672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 656/1326] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:52.532902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd64407-3e83-4947-b14f-534ee23b137c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:52.534445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9dd64407-3e83-4947-b14f-534ee23b137c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:52.559543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd64407-3e83-4947-b14f-534ee23b137c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:52.569547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:52.570572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:52.575812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:52.576039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] PATH_INFO: `/v3/projects/cee0a6ee88fa492ab67c8913c5b71117` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:52.593028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee0a6ee88fa492ab67c8913c5b71117)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:52.594239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: WARNING py.warnings [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 14:40:52.594239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: warnings.warn(msg) Oct 20 14:40:52.594239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: Oct 20 14:40:52.600752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:52.761484 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed3bc7-4bca-4375-9556-89264ae4eb28 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:52.763626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-84ed3bc7-4bca-4375-9556-89264ae4eb28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:52.819598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed3bc7-4bca-4375-9556-89264ae4eb28 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:52.846611 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee0a6ee88fa492ab67c8913c5b71117', 'request_id': 'req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:52.864753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee0a6ee88fa492ab67c8913c5b71117', 'request_id': 'req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:52.890842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee0a6ee88fa492ab67c8913c5b71117', 'request_id': 'req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:52.941331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] The token cache is being invalidate because project cee0a6ee88fa492ab67c8913c5b71117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:40:52.941677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:52.944282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:52.948921 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 671/1327] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:40:52 2022] DELETE /identity/v3/projects/cee0a6ee88fa492ab67c8913c5b71117 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:52.959991 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:52.963692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:52.968285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:53.010482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:53.011397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:53.013522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:53.026874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] PATH_INFO: `/v3/projects/1ebb3ea35947435b827833547073f0a5` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:53.032653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ebb3ea35947435b827833547073f0a5)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:53.037555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:53.185354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5a8f1-7c85-4182-bc1e-ebc31a3270f1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:53.185354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:53.185354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:53.185354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:53.187380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] PATH_INFO: `/v3/users/abe8ee85ccbd4736b3a82073517450f4` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:53.187380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe8ee85ccbd4736b3a82073517450f4)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:53.187380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:53.190568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ebb3ea35947435b827833547073f0a5', 'request_id': 'req-96db0ba6-db25-4090-9552-afe27e5965a0'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.208726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ebb3ea35947435b827833547073f0a5', 'request_id': 'req-96db0ba6-db25-4090-9552-afe27e5965a0'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.220600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:53.223652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ebb3ea35947435b827833547073f0a5', 'request_id': 'req-96db0ba6-db25-4090-9552-afe27e5965a0'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.270243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] The token cache is being invalidate because project 1ebb3ea35947435b827833547073f0a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:40:53.274996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.279649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.284904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe8ee85ccbd4736b3a82073517450f4'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.291405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 657/1328] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:40:52 2022] DELETE /identity/v3/projects/1ebb3ea35947435b827833547073f0a5 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:40:53.295834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:53.305256 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe8ee85ccbd4736b3a82073517450f4'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.306243 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe8ee85ccbd4736b3a82073517450f4'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.307282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe8ee85ccbd4736b3a82073517450f4'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.308226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:53.334699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe8ee85ccbd4736b3a82073517450f4'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.377577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 672/1329] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:40:52 2022] DELETE /identity/v3/users/abe8ee85ccbd4736b3a82073517450f4 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:53.479579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db0ba6-db25-4090-9552-afe27e5965a0 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:53.596814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:53.599701 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:53.603416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:53.604947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] PATH_INFO: `/v3/users/b19ac02af2f34adb8ceb8d348de05173` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:53.609653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b19ac02af2f34adb8ceb8d348de05173)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:53.615010 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:53.690493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:53.715231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b19ac02af2f34adb8ceb8d348de05173'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.732672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b19ac02af2f34adb8ceb8d348de05173'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.733927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b19ac02af2f34adb8ceb8d348de05173'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.734959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b19ac02af2f34adb8ceb8d348de05173'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.750115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-84205e02-c431-4f80-a2b2-e272148fdc1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b19ac02af2f34adb8ceb8d348de05173'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:53.800114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 658/1330] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:40:53 2022] DELETE /identity/v3/users/b19ac02af2f34adb8ceb8d348de05173 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:55.051426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:55.053639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:55.088747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8884541-eb77-48f5-9186-0df72a454c0f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:55.289128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:55.290021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:55.290379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:55.290786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] PATH_INFO: `/v3/projects/150a7512d51b4ad0bf949e0f625d9a10` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:55.295545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=150a7512d51b4ad0bf949e0f625d9a10)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:55.297747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:55.460684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '150a7512d51b4ad0bf949e0f625d9a10', 'request_id': 'req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:55.476141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '150a7512d51b4ad0bf949e0f625d9a10', 'request_id': 'req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:55.499987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '150a7512d51b4ad0bf949e0f625d9a10', 'request_id': 'req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:55.539815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] The token cache is being invalidate because project 150a7512d51b4ad0bf949e0f625d9a10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:40:55.540761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:55.544667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-16bb0f90-8e6e-4fd5-bba1-6405b1885cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:55.551954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 673/1331] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:40:55 2022] DELETE /identity/v3/projects/150a7512d51b4ad0bf949e0f625d9a10 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:55.553826 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e4f8e-9aab-45ac-a4dc-6a528cf9f0b8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:55.554679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e4f8e-9aab-45ac-a4dc-6a528cf9f0b8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:55.555148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e4f8e-9aab-45ac-a4dc-6a528cf9f0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:55.616209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-9d0e4f8e-9aab-45ac-a4dc-6a528cf9f0b8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:55.621859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9d0e4f8e-9aab-45ac-a4dc-6a528cf9f0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:55.808529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 659/1332] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:55.818487 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0ebc1-2b3f-40ba-bc05-502fee27a4d4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:55.818857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0ebc1-2b3f-40ba-bc05-502fee27a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:55.819210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0ebc1-2b3f-40ba-bc05-502fee27a4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:55.862743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0bd0ebc1-2b3f-40ba-bc05-502fee27a4d4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:55.864073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0bd0ebc1-2b3f-40ba-bc05-502fee27a4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:55.920171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 674/1333] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:55.920645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e4f8e-9aab-45ac-a4dc-6a528cf9f0b8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:55.938858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e4f8e-9aab-45ac-a4dc-6a528cf9f0b8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:55.968069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ae385-5b5e-4bf4-b58c-34e3b53e9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:55.969283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ae385-5b5e-4bf4-b58c-34e3b53e9ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:55.972458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ae385-5b5e-4bf4-b58c-34e3b53e9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:55.973657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ae385-5b5e-4bf4-b58c-34e3b53e9ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:55.989091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ae385-5b5e-4bf4-b58c-34e3b53e9ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:55.990872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ae385-5b5e-4bf4-b58c-34e3b53e9ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:56.007136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ae385-5b5e-4bf4-b58c-34e3b53e9ea7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:56.049153 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 660/1334] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:56.055674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0ebc1-2b3f-40ba-bc05-502fee27a4d4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:56.072536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0ebc1-2b3f-40ba-bc05-502fee27a4d4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:56.086950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff20b8-8a48-4321-9046-134a9d8129a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:56.087882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff20b8-8a48-4321-9046-134a9d8129a8 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:56.088419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff20b8-8a48-4321-9046-134a9d8129a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:56.088916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff20b8-8a48-4321-9046-134a9d8129a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:56.090371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff20b8-8a48-4321-9046-134a9d8129a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:56.092926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff20b8-8a48-4321-9046-134a9d8129a8 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:56.162282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:56.208021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 675/1335] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:56 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:56.209620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ae385-5b5e-4bf4-b58c-34e3b53e9ea7 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:56.221025 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ae385-5b5e-4bf4-b58c-34e3b53e9ea7 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:56.401192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5eee8a-c685-4991-87ea-3d5edd5dc645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:56.401877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eee8a-c685-4991-87ea-3d5edd5dc645 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:56.402767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eee8a-c685-4991-87ea-3d5edd5dc645 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:56.403282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eee8a-c685-4991-87ea-3d5edd5dc645 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:56.404124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5eee8a-c685-4991-87ea-3d5edd5dc645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:56.406390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5eee8a-c685-4991-87ea-3d5edd5dc645 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:56.454104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 661/1336] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:56 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:56.458645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff20b8-8a48-4321-9046-134a9d8129a8 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:56.473966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff20b8-8a48-4321-9046-134a9d8129a8 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:56.495973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55efaed-4f0c-48af-957a-26f5fafa3e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:56.505098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55efaed-4f0c-48af-957a-26f5fafa3e27 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:56.513858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55efaed-4f0c-48af-957a-26f5fafa3e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:56.515566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55efaed-4f0c-48af-957a-26f5fafa3e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:56.521156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55efaed-4f0c-48af-957a-26f5fafa3e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:56.529063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5eee8a-c685-4991-87ea-3d5edd5dc645 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:56.532949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bf5eee8a-c685-4991-87ea-3d5edd5dc645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:56.538580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55efaed-4f0c-48af-957a-26f5fafa3e27 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:56.567678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5eee8a-c685-4991-87ea-3d5edd5dc645 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:56.596248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:56.601183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:56.608116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:56.608664 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] PATH_INFO: `/v3/projects/6498c6e729ee4c8f818a3a18d6d0790c` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:56.608929 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 676/1337] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:56.610063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55efaed-4f0c-48af-957a-26f5fafa3e27 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:56.611210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6498c6e729ee4c8f818a3a18d6d0790c)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:56.614522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:56.689580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55efaed-4f0c-48af-957a-26f5fafa3e27 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:56.706316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5f9ce-03f1-4916-aad2-32120860ff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:56.708513 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5f9ce-03f1-4916-aad2-32120860ff24 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:56.709947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5f9ce-03f1-4916-aad2-32120860ff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:56.710447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5f9ce-03f1-4916-aad2-32120860ff24 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:56.711330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5f9ce-03f1-4916-aad2-32120860ff24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:56.718904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5f9ce-03f1-4916-aad2-32120860ff24 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:56.738904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:56.767002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 677/1338] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:56.769039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5f9ce-03f1-4916-aad2-32120860ff24 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:56.821784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5f9ce-03f1-4916-aad2-32120860ff24 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:56.839502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6498c6e729ee4c8f818a3a18d6d0790c', 'request_id': 'req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:56.878154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6498c6e729ee4c8f818a3a18d6d0790c', 'request_id': 'req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:56.915679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6498c6e729ee4c8f818a3a18d6d0790c', 'request_id': 'req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:56.970597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] The token cache is being invalidate because project 6498c6e729ee4c8f818a3a18d6d0790c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:40:56.971056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:56.978930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f0f4311f-f346-4ab8-b2db-2797ba2cfa63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:56.988618 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 662/1339] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:40:56 2022] DELETE /identity/v3/projects/6498c6e729ee4c8f818a3a18d6d0790c => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:56.992785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934141-b2dd-45e8-96d4-83fdacb68c79 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:56.995231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934141-b2dd-45e8-96d4-83fdacb68c79 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:56.995522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934141-b2dd-45e8-96d4-83fdacb68c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.030594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b0fa4-7f60-47dc-ae30-84353d44a340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:57.033486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0fa4-7f60-47dc-ae30-84353d44a340 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.036695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0fa4-7f60-47dc-ae30-84353d44a340 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.037903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0fa4-7f60-47dc-ae30-84353d44a340 admin admin] PATH_INFO: `/v3/projects/d2b475ae06da45b983011f02e8db23c0/users/8f79f5fb6218464aa2bdef26aa5c21da/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.090789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d7934141-b2dd-45e8-96d4-83fdacb68c79 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:57.095417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d7934141-b2dd-45e8-96d4-83fdacb68c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:57.106096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b0fa4-7f60-47dc-ae30-84353d44a340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b475ae06da45b983011f02e8db23c0, user_id=8f79f5fb6218464aa2bdef26aa5c21da, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:57.112505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b0fa4-7f60-47dc-ae30-84353d44a340 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:57.122481 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 663/1340] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:57.135072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc787b-c3d4-44da-b1f6-c5092ecee9d1 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.135820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc787b-c3d4-44da-b1f6-c5092ecee9d1 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.136477 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc787b-c3d4-44da-b1f6-c5092ecee9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.175192 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 678/1341] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:56 2022] PUT /identity/v3/projects/d2b475ae06da45b983011f02e8db23c0/users/8f79f5fb6218464aa2bdef26aa5c21da/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:57.178423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406630-07a4-4228-9d93-82dbb61fb28d None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.180497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406630-07a4-4228-9d93-82dbb61fb28d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.181253 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406630-07a4-4228-9d93-82dbb61fb28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.212067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-afdc787b-c3d4-44da-b1f6-c5092ecee9d1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:57.218069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-afdc787b-c3d4-44da-b1f6-c5092ecee9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:57.260213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-5e406630-07a4-4228-9d93-82dbb61fb28d None None] MFA Rules not processed for user `8f79f5fb6218464aa2bdef26aa5c21da`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:57.262861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5e406630-07a4-4228-9d93-82dbb61fb28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:57.398710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 664/1342] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:57.404916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc787b-c3d4-44da-b1f6-c5092ecee9d1 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:57.437739 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc787b-c3d4-44da-b1f6-c5092ecee9d1 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:57.445532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 679/1343] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:57.454132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e406630-07a4-4228-9d93-82dbb61fb28d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:57.456922 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5e406630-07a4-4228-9d93-82dbb61fb28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:57.462035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcac16-af11-4eae-b22b-1bbd0f3e8e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:57.462678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcac16-af11-4eae-b22b-1bbd0f3e8e0d admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.462961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcac16-af11-4eae-b22b-1bbd0f3e8e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.463237 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcac16-af11-4eae-b22b-1bbd0f3e8e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.463900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcac16-af11-4eae-b22b-1bbd0f3e8e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:40:57.464208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcac16-af11-4eae-b22b-1bbd0f3e8e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:57.466622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcac16-af11-4eae-b22b-1bbd0f3e8e0d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:57.489774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e406630-07a4-4228-9d93-82dbb61fb28d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:57.499465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 665/1344] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:40:57.504282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685760f-ad2a-471a-82e6-161582bb8aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:57.504503 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcac16-af11-4eae-b22b-1bbd0f3e8e0d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:57.505251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685760f-ad2a-471a-82e6-161582bb8aea admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.505911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685760f-ad2a-471a-82e6-161582bb8aea admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.506345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685760f-ad2a-471a-82e6-161582bb8aea admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.507128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685760f-ad2a-471a-82e6-161582bb8aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:57.510885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685760f-ad2a-471a-82e6-161582bb8aea admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:57.527770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcac16-af11-4eae-b22b-1bbd0f3e8e0d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:57.541305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457f304-ec35-4b39-a010-b05557009fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:57.542139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457f304-ec35-4b39-a010-b05557009fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.544519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457f304-ec35-4b39-a010-b05557009fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.544916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457f304-ec35-4b39-a010-b05557009fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.549522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457f304-ec35-4b39-a010-b05557009fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:57.554667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457f304-ec35-4b39-a010-b05557009fb1 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:57.572006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 680/1345] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:57 2022] POST /identity/v3/projects => generated 361 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:57.576849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685760f-ad2a-471a-82e6-161582bb8aea admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:57.608156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 666/1346] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:57 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:57.609390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457f304-ec35-4b39-a010-b05557009fb1 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:57.657986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:57.735612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685760f-ad2a-471a-82e6-161582bb8aea admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:57.788729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457f304-ec35-4b39-a010-b05557009fb1 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:57.803722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a2b05-64b3-4437-a166-325d8d695993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:57.804373 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a2b05-64b3-4437-a166-325d8d695993 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.804719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a2b05-64b3-4437-a166-325d8d695993 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.804983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a2b05-64b3-4437-a166-325d8d695993 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.805623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a2b05-64b3-4437-a166-325d8d695993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:57.807612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a2b05-64b3-4437-a166-325d8d695993 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:57.845573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:57.857955 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 667/1347] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:57 2022] POST /identity/v3/users => generated 291 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:57.859608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a2b05-64b3-4437-a166-325d8d695993 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:57.860488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ee866-f7d4-4904-b503-052c1bf5136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:57.867746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ee866-f7d4-4904-b503-052c1bf5136c admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.868172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ee866-f7d4-4904-b503-052c1bf5136c admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.868481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ee866-f7d4-4904-b503-052c1bf5136c admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.869244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ee866-f7d4-4904-b503-052c1bf5136c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:57.875182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ee866-f7d4-4904-b503-052c1bf5136c admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:57.890725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a2b05-64b3-4437-a166-325d8d695993 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:57.899328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007d157-b635-4850-b233-6973cfd21bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:57.900100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007d157-b635-4850-b233-6973cfd21bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.900278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007d157-b635-4850-b233-6973cfd21bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.900640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007d157-b635-4850-b233-6973cfd21bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.901190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007d157-b635-4850-b233-6973cfd21bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:57.903738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007d157-b635-4850-b233-6973cfd21bb4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:57.927933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 681/1348] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:57 2022] POST /identity/v3/users => generated 287 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:40:57.930500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ee866-f7d4-4904-b503-052c1bf5136c admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:57.947639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 668/1349] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:57.947639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007d157-b635-4850-b233-6973cfd21bb4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:57.959077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ee866-f7d4-4904-b503-052c1bf5136c admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:57.970518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007d157-b635-4850-b233-6973cfd21bb4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:57.975577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ddf8f-7f62-4b5b-acf7-212372d47f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:57.976206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ddf8f-7f62-4b5b-acf7-212372d47f72 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.976509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ddf8f-7f62-4b5b-acf7-212372d47f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.976779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ddf8f-7f62-4b5b-acf7-212372d47f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.977425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ddf8f-7f62-4b5b-acf7-212372d47f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:57.979032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ddf8f-7f62-4b5b-acf7-212372d47f72 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:57.984923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fdf23-235e-40f9-a7f5-14f0e3086848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:57.985566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fdf23-235e-40f9-a7f5-14f0e3086848 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:57.986064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fdf23-235e-40f9-a7f5-14f0e3086848 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:57.986369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fdf23-235e-40f9-a7f5-14f0e3086848 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:57.987287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fdf23-235e-40f9-a7f5-14f0e3086848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:57.988737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fdf23-235e-40f9-a7f5-14f0e3086848 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:58.010751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 682/1350] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:58.012727 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ddf8f-7f62-4b5b-acf7-212372d47f72 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:58.020447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 669/1351] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:58.023010 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fdf23-235e-40f9-a7f5-14f0e3086848 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:58.038874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ddf8f-7f62-4b5b-acf7-212372d47f72 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:58.045197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fdf23-235e-40f9-a7f5-14f0e3086848 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:58.050953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d860073-0e85-44b5-8666-ef49ef5aeb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:58.051692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d860073-0e85-44b5-8666-ef49ef5aeb5d admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:58.051912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d860073-0e85-44b5-8666-ef49ef5aeb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:58.052195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d860073-0e85-44b5-8666-ef49ef5aeb5d admin admin] PATH_INFO: `/v3/projects/be95a4f8cd6441e380dd6f303027cf04/users/f8af325a9fac4d40b84c08bbb67c037e/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:58.059481 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9bfdb-9db9-453a-a968-8bbf1c13d1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:58.060821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9bfdb-9db9-453a-a968-8bbf1c13d1fd admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:58.061106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9bfdb-9db9-453a-a968-8bbf1c13d1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:58.061369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9bfdb-9db9-453a-a968-8bbf1c13d1fd admin admin] PATH_INFO: `/v3/projects/9ead5e2e710f4524b504763d5f809de1/users/66ed1729148f4417998c294d0aae2ee0/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:58.086591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d860073-0e85-44b5-8666-ef49ef5aeb5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be95a4f8cd6441e380dd6f303027cf04, user_id=f8af325a9fac4d40b84c08bbb67c037e, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:58.089313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d860073-0e85-44b5-8666-ef49ef5aeb5d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:58.098006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9bfdb-9db9-453a-a968-8bbf1c13d1fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ead5e2e710f4524b504763d5f809de1, user_id=66ed1729148f4417998c294d0aae2ee0, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:58.104924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9bfdb-9db9-453a-a968-8bbf1c13d1fd admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:58.113762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 683/1352] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:58 2022] PUT /identity/v3/projects/be95a4f8cd6441e380dd6f303027cf04/users/f8af325a9fac4d40b84c08bbb67c037e/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:58.117540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d860073-0e85-44b5-8666-ef49ef5aeb5d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:58.142918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 670/1353] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:58 2022] PUT /identity/v3/projects/9ead5e2e710f4524b504763d5f809de1/users/66ed1729148f4417998c294d0aae2ee0/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:58.154870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3464ae7-70f7-4cbe-999a-752932dfd37f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:58.155309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3464ae7-70f7-4cbe-999a-752932dfd37f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:58.155948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3464ae7-70f7-4cbe-999a-752932dfd37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:58.218675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-c3464ae7-70f7-4cbe-999a-752932dfd37f None None] MFA Rules not processed for user `66ed1729148f4417998c294d0aae2ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:58.221251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c3464ae7-70f7-4cbe-999a-752932dfd37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:58.264627 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d860073-0e85-44b5-8666-ef49ef5aeb5d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:58.341984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:58.391331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 671/1354] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:58.392547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3464ae7-70f7-4cbe-999a-752932dfd37f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:58.418790 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350541a2-aa16-4d9b-b361-8644ec7f4548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:58.419501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350541a2-aa16-4d9b-b361-8644ec7f4548 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:58.419856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350541a2-aa16-4d9b-b361-8644ec7f4548 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:58.420264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350541a2-aa16-4d9b-b361-8644ec7f4548 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:58.421008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350541a2-aa16-4d9b-b361-8644ec7f4548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:58.426131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350541a2-aa16-4d9b-b361-8644ec7f4548 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:58.449927 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 684/1355] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:40:58.485069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350541a2-aa16-4d9b-b361-8644ec7f4548 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:58.509988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350541a2-aa16-4d9b-b361-8644ec7f4548 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:58.527869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d610c-bc52-48a4-89f4-dd7c16de8170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:58.528808 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d610c-bc52-48a4-89f4-dd7c16de8170 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:58.529384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d610c-bc52-48a4-89f4-dd7c16de8170 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:58.529900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d610c-bc52-48a4-89f4-dd7c16de8170 admin admin] PATH_INFO: `/v3/projects/be95a4f8cd6441e380dd6f303027cf04/users/f8af325a9fac4d40b84c08bbb67c037e/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:58.534181 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d610c-bc52-48a4-89f4-dd7c16de8170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be95a4f8cd6441e380dd6f303027cf04, user_id=f8af325a9fac4d40b84c08bbb67c037e, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:58.553976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d610c-bc52-48a4-89f4-dd7c16de8170 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:58.594548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 685/1356] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:58 2022] PUT /identity/v3/projects/be95a4f8cd6441e380dd6f303027cf04/users/f8af325a9fac4d40b84c08bbb67c037e/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:58.597571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d610c-bc52-48a4-89f4-dd7c16de8170 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:58.599778 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-127d610c-bc52-48a4-89f4-dd7c16de8170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:58.624247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3464ae7-70f7-4cbe-999a-752932dfd37f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:58.774195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:58.809705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d610c-bc52-48a4-89f4-dd7c16de8170 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:58.822880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:58.823541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:58.823818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:58.824163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] PATH_INFO: `/v3/users/f14f13f618ac44e9a1f463be2852b368` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:58.826527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d931410-312c-4ec3-b4d8-f537e88d8ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:58.827156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d931410-312c-4ec3-b4d8-f537e88d8ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:58.827431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d931410-312c-4ec3-b4d8-f537e88d8ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:58.827701 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d931410-312c-4ec3-b4d8-f537e88d8ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:58.828814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f14f13f618ac44e9a1f463be2852b368)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:58.830797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:58.834291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d931410-312c-4ec3-b4d8-f537e88d8ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:58.835278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d931410-312c-4ec3-b4d8-f537e88d8ea4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:58.900432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 672/1357] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:40:58 2022] POST /identity/v3/projects => generated 383 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:58.900432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57a0f6-ba90-4d88-b046-d5f4b6477424 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:58.900432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57a0f6-ba90-4d88-b046-d5f4b6477424 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:58.900777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57a0f6-ba90-4d88-b046-d5f4b6477424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:58.936732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f14f13f618ac44e9a1f463be2852b368'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:58.960183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f14f13f618ac44e9a1f463be2852b368'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:58.963068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f14f13f618ac44e9a1f463be2852b368'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:58.964882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f14f13f618ac44e9a1f463be2852b368'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:58.981673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ee57a0f6-ba90-4d88-b046-d5f4b6477424 None None] MFA Rules not processed for user `f8af325a9fac4d40b84c08bbb67c037e`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:58.983977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f14f13f618ac44e9a1f463be2852b368'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:40:58.990089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ee57a0f6-ba90-4d88-b046-d5f4b6477424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:59.028730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 686/1358] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:40:58 2022] DELETE /identity/v3/users/f14f13f618ac44e9a1f463be2852b368 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:59.033865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:59.038251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:59.161790 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:59.250746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 673/1359] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:59.254735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3187bc-1b12-40cc-9f48-f4904f90e070 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:59.255024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3187bc-1b12-40cc-9f48-f4904f90e070 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:59.255314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3187bc-1b12-40cc-9f48-f4904f90e070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:59.260989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648261b-e410-423e-be20-ee3f826b38fc admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:59.276957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf5e55-772d-4c10-821b-dbca38e8e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:59.277649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf5e55-772d-4c10-821b-dbca38e8e9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:59.277931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf5e55-772d-4c10-821b-dbca38e8e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:59.278205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf5e55-772d-4c10-821b-dbca38e8e9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:59.278841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf5e55-772d-4c10-821b-dbca38e8e9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:59.289050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf5e55-772d-4c10-821b-dbca38e8e9a2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:59.310855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6f3187bc-1b12-40cc-9f48-f4904f90e070 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:40:59.315367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6f3187bc-1b12-40cc-9f48-f4904f90e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:59.336407 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 674/1360] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:40:59.346890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3187bc-1b12-40cc-9f48-f4904f90e070 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:59.357524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6f3187bc-1b12-40cc-9f48-f4904f90e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:59.365452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 687/1361] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:40:59 2022] POST /identity/v3/users => generated 298 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:40:59.368287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf5e55-772d-4c10-821b-dbca38e8e9a2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:59.443834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf5e55-772d-4c10-821b-dbca38e8e9a2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:59.465517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c46c54-0ebc-4a97-bb70-3d5701476b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:59.466915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c46c54-0ebc-4a97-bb70-3d5701476b58 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:59.474006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c46c54-0ebc-4a97-bb70-3d5701476b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:59.482927 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c46c54-0ebc-4a97-bb70-3d5701476b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:59.485058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c46c54-0ebc-4a97-bb70-3d5701476b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:59.488542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c46c54-0ebc-4a97-bb70-3d5701476b58 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:59.532254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 688/1362] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:40:59.538843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c46c54-0ebc-4a97-bb70-3d5701476b58 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:59.544484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b4c46c54-0ebc-4a97-bb70-3d5701476b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:40:59.657609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3187bc-1b12-40cc-9f48-f4904f90e070 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:59.664100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c747a-3eea-4321-b945-f4fda000ecf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:59.665170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c747a-3eea-4321-b945-f4fda000ecf7 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:59.665641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c747a-3eea-4321-b945-f4fda000ecf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:59.666125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c747a-3eea-4321-b945-f4fda000ecf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:59.814700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c46c54-0ebc-4a97-bb70-3d5701476b58 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:59.827819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fad050-4cec-49f1-9346-1554f0d061fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:59.829334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fad050-4cec-49f1-9346-1554f0d061fd service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:59.829334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fad050-4cec-49f1-9346-1554f0d061fd service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:59.829334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fad050-4cec-49f1-9346-1554f0d061fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:59.851219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fad050-4cec-49f1-9346-1554f0d061fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:59.852720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fad050-4cec-49f1-9346-1554f0d061fd service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:59.853010 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c747a-3eea-4321-b945-f4fda000ecf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:59.854466 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c747a-3eea-4321-b945-f4fda000ecf7 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:59.872997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 675/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:59 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:59.875126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c747a-3eea-4321-b945-f4fda000ecf7 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:59.879182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 689/1364] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:40:59.882657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:40:59.889579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c747a-3eea-4321-b945-f4fda000ecf7 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:40:59.896461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54419938-111b-4cfb-bfe9-5e64c451e8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:40:59.897531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54419938-111b-4cfb-bfe9-5e64c451e8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:40:59.898036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54419938-111b-4cfb-bfe9-5e64c451e8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:40:59.898516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54419938-111b-4cfb-bfe9-5e64c451e8f4 admin admin] PATH_INFO: `/v3/projects/d867bf225fff4c568bac88091cbb129b/users/aa5ecc0bd4de48b3876ea9657065187e/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:40:59.924086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54419938-111b-4cfb-bfe9-5e64c451e8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d867bf225fff4c568bac88091cbb129b, user_id=aa5ecc0bd4de48b3876ea9657065187e, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:40:59.927404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54419938-111b-4cfb-bfe9-5e64c451e8f4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:40:59.963855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 676/1365] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:40:59 2022] PUT /identity/v3/projects/d867bf225fff4c568bac88091cbb129b/users/aa5ecc0bd4de48b3876ea9657065187e/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:40:59.968602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fad050-4cec-49f1-9346-1554f0d061fd service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:40:59.983340 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fad050-4cec-49f1-9346-1554f0d061fd service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:00.091727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54419938-111b-4cfb-bfe9-5e64c451e8f4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:00.165180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fe204-ca8d-43ce-a619-95f161194296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:00.166564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fe204-ca8d-43ce-a619-95f161194296 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:00.166995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fe204-ca8d-43ce-a619-95f161194296 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:00.167431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fe204-ca8d-43ce-a619-95f161194296 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:00.168430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fe204-ca8d-43ce-a619-95f161194296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:00.170277 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fe204-ca8d-43ce-a619-95f161194296 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:00.191038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 690/1366] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:00.194906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fe204-ca8d-43ce-a619-95f161194296 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:00.231416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fe204-ca8d-43ce-a619-95f161194296 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:00.250705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54419938-111b-4cfb-bfe9-5e64c451e8f4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:00.253106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdddb7-3c1e-4981-833f-0a2f8cf9a8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:00.254405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdddb7-3c1e-4981-833f-0a2f8cf9a8d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:00.255715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdddb7-3c1e-4981-833f-0a2f8cf9a8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:00.259478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:00.259985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdddb7-3c1e-4981-833f-0a2f8cf9a8d0 admin admin] PATH_INFO: `/v3/projects/d867bf225fff4c568bac88091cbb129b/users/aa5ecc0bd4de48b3876ea9657065187e/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:00.263425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:00.263633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:00.263918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:00.266805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:00.267985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdddb7-3c1e-4981-833f-0a2f8cf9a8d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d867bf225fff4c568bac88091cbb129b, user_id=aa5ecc0bd4de48b3876ea9657065187e, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:00.275265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdddb7-3c1e-4981-833f-0a2f8cf9a8d0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:00.315928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 691/1367] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:00 2022] PUT /identity/v3/projects/d867bf225fff4c568bac88091cbb129b/users/aa5ecc0bd4de48b3876ea9657065187e/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:00.328188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91098b-5077-4d62-a8e2-e5007a291289 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:00.328694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91098b-5077-4d62-a8e2-e5007a291289 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:00.328981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91098b-5077-4d62-a8e2-e5007a291289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:00.398362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1d91098b-5077-4d62-a8e2-e5007a291289 None None] MFA Rules not processed for user `aa5ecc0bd4de48b3876ea9657065187e`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:00.400324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1d91098b-5077-4d62-a8e2-e5007a291289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:00.406089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:00.407615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:00.434605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 677/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:00.462027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:00.567505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 692/1369] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:00.581062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:00.582514 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:00.602189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779df2f1-2dcc-41b0-ae91-c69b522d82dd service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:00.692141 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7504de9-83ad-452e-a865-65c2d696e8bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:00.692827 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7504de9-83ad-452e-a865-65c2d696e8bd service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:00.694312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7504de9-83ad-452e-a865-65c2d696e8bd service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:00.694312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7504de9-83ad-452e-a865-65c2d696e8bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:00.759580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7504de9-83ad-452e-a865-65c2d696e8bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:00.763716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7504de9-83ad-452e-a865-65c2d696e8bd service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:00.829836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 678/1370] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:01.019165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91098b-5077-4d62-a8e2-e5007a291289 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:01.037510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91098b-5077-4d62-a8e2-e5007a291289 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:01.047912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8198b-e1fc-4051-be0f-4c597060360e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:01.051052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8198b-e1fc-4051-be0f-4c597060360e service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:01.051532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8198b-e1fc-4051-be0f-4c597060360e service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:01.052087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8198b-e1fc-4051-be0f-4c597060360e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:01.054249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3ff8198b-e1fc-4051-be0f-4c597060360e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:01.204219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8198b-e1fc-4051-be0f-4c597060360e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:01.206070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8198b-e1fc-4051-be0f-4c597060360e service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:01.233266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 693/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:01.571195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7504de9-83ad-452e-a865-65c2d696e8bd service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:01.575456 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a7504de9-83ad-452e-a865-65c2d696e8bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:01.612724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed2d69-60cd-47fc-a381-044aba3a9ba2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:01.612966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed2d69-60cd-47fc-a381-044aba3a9ba2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:01.614667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed2d69-60cd-47fc-a381-044aba3a9ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:01.650573 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0fed2d69-60cd-47fc-a381-044aba3a9ba2 None None] MFA Rules not processed for user `812daba513084a55badecfcde843a6ff`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:01.652744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0fed2d69-60cd-47fc-a381-044aba3a9ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:01.723567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:01.767636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 694/1372] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:01.769359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4282b1-6954-4109-9dd7-93837b7b3b1d None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:01.769742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4282b1-6954-4109-9dd7-93837b7b3b1d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:01.770127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4282b1-6954-4109-9dd7-93837b7b3b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:01.801711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-bb4282b1-6954-4109-9dd7-93837b7b3b1d None None] MFA Rules not processed for user `812daba513084a55badecfcde843a6ff`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:01.802974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bb4282b1-6954-4109-9dd7-93837b7b3b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:01.806720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7504de9-83ad-452e-a865-65c2d696e8bd service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:01.813463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 695/1373] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:01.827762 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:01.830373 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:01.831852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:01.834761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] PATH_INFO: `/v3/users/6b5c1dbc28bc4c009f2cc2c45dcea2d9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:01.840266 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5c1dbc28bc4c009f2cc2c45dcea2d9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:01.840851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4282b1-6954-4109-9dd7-93837b7b3b1d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:01.843117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:01.856282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bb4282b1-6954-4109-9dd7-93837b7b3b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:01.979863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5c1dbc28bc4c009f2cc2c45dcea2d9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:02.015400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5c1dbc28bc4c009f2cc2c45dcea2d9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:02.016377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5c1dbc28bc4c009f2cc2c45dcea2d9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:02.017170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5c1dbc28bc4c009f2cc2c45dcea2d9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:02.044781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bf607a3b-22ab-409e-843b-dbfdda25a395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5c1dbc28bc4c009f2cc2c45dcea2d9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:02.061322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:02.094536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4282b1-6954-4109-9dd7-93837b7b3b1d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:02.111677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:02.112665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:02.114358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:02.116820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:02.123600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:02.143079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 679/1374] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:01 2022] DELETE /identity/v3/users/6b5c1dbc28bc4c009f2cc2c45dcea2d9 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:02.151533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22199106-2ad7-44af-8768-0b18f3756509 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:02.151830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22199106-2ad7-44af-8768-0b18f3756509 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:02.152834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22199106-2ad7-44af-8768-0b18f3756509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:02.161852 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:02.164280 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:02.200532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 696/1375] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:02.242691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-22199106-2ad7-44af-8768-0b18f3756509 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:02.244051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-22199106-2ad7-44af-8768-0b18f3756509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:02.512262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 680/1376] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:02.584402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:02.584402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:02.692374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ef8de-11e9-4a63-ad84-b6d35339b1cd service nova] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:02.823089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9aff2-b693-4949-831d-b03182a42d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:02.824237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9aff2-b693-4949-831d-b03182a42d6e service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:02.824739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9aff2-b693-4949-831d-b03182a42d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:02.825017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9aff2-b693-4949-831d-b03182a42d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:02.845004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9aff2-b693-4949-831d-b03182a42d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:02.846463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9aff2-b693-4949-831d-b03182a42d6e service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:02.871709 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 697/1377] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:03.690498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22199106-2ad7-44af-8768-0b18f3756509 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:03.692250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-22199106-2ad7-44af-8768-0b18f3756509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:03.719152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22199106-2ad7-44af-8768-0b18f3756509 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:03.728274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:03.729208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:03.729731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:03.730442 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] PATH_INFO: `/v3/projects/22dfe6d702cb4ad690238da1a6c20163` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:03.732606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=22dfe6d702cb4ad690238da1a6c20163)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:03.735202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:03.790128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a14e14-4a87-4bbf-b02a-53ad5ec44147 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:03.790668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a14e14-4a87-4bbf-b02a-53ad5ec44147 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:03.790668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a14e14-4a87-4bbf-b02a-53ad5ec44147 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:03.792547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 698/1378] 10.222.0.13 () {54 vars in 912 bytes} [Thu Oct 20 14:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:41:03.804774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7b32a-f2e0-4cca-a79b-d2912cf12a0d None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:03.805446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7b32a-f2e0-4cca-a79b-d2912cf12a0d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:03.806685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7b32a-f2e0-4cca-a79b-d2912cf12a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:03.866350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dfe6d702cb4ad690238da1a6c20163', 'request_id': 'req-2285134e-5891-4ba4-8dc6-0416557f20bb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:03.869391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:03.881793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dfe6d702cb4ad690238da1a6c20163', 'request_id': 'req-2285134e-5891-4ba4-8dc6-0416557f20bb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:03.902850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dfe6d702cb4ad690238da1a6c20163', 'request_id': 'req-2285134e-5891-4ba4-8dc6-0416557f20bb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:03.916391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-bff7b32a-f2e0-4cca-a79b-d2912cf12a0d None None] MFA Rules not processed for user `ae4b023dbf114bf98d0e38ea7fec94fa`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:03.918480 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bff7b32a-f2e0-4cca-a79b-d2912cf12a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:03.923057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] The token cache is being invalidate because project 22dfe6d702cb4ad690238da1a6c20163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:03.923587 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:03.924697 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2285134e-5891-4ba4-8dc6-0416557f20bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:03.942853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 681/1379] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:03 2022] DELETE /identity/v3/projects/22dfe6d702cb4ad690238da1a6c20163 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:03.944888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111fc7a-e097-4438-bf7d-47b642e883a2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:03.944888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111fc7a-e097-4438-bf7d-47b642e883a2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:03.945208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111fc7a-e097-4438-bf7d-47b642e883a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:04.005443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-3111fc7a-e097-4438-bf7d-47b642e883a2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:04.006674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3111fc7a-e097-4438-bf7d-47b642e883a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:04.150369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 682/1380] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:04.159594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf3fa4-8b55-45f4-b415-bb4b74485114 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:04.160757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf3fa4-8b55-45f4-b415-bb4b74485114 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:04.161333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf3fa4-8b55-45f4-b415-bb4b74485114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:04.163377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 699/1381] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:41:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:04.234015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-a0bf3fa4-8b55-45f4-b415-bb4b74485114 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:04.236961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a0bf3fa4-8b55-45f4-b415-bb4b74485114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:04.250833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 683/1382] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:04.254873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7b32a-f2e0-4cca-a79b-d2912cf12a0d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:04.274522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7b32a-f2e0-4cca-a79b-d2912cf12a0d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:04.295669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301cbca-650e-4331-bbec-3d616ad21446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:04.296314 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301cbca-650e-4331-bbec-3d616ad21446 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:04.296664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301cbca-650e-4331-bbec-3d616ad21446 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:04.296936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301cbca-650e-4331-bbec-3d616ad21446 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:04.297610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301cbca-650e-4331-bbec-3d616ad21446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:04.297912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301cbca-650e-4331-bbec-3d616ad21446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:04.299912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301cbca-650e-4331-bbec-3d616ad21446 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:04.319824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf3fa4-8b55-45f4-b415-bb4b74485114 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:04.321615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a0bf3fa4-8b55-45f4-b415-bb4b74485114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:04.345127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 700/1383] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:04.351705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf3fa4-8b55-45f4-b415-bb4b74485114 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:04.352015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301cbca-650e-4331-bbec-3d616ad21446 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:04.359365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f57517-3536-4d9d-ab28-74bdf46681de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:04.361330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f57517-3536-4d9d-ab28-74bdf46681de service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:04.361791 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f57517-3536-4d9d-ab28-74bdf46681de service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:04.362182 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f57517-3536-4d9d-ab28-74bdf46681de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:04.382289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301cbca-650e-4331-bbec-3d616ad21446 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:04.387871 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8ff59-92d1-469b-85de-62ba50bf240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:04.388529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8ff59-92d1-469b-85de-62ba50bf240a admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:04.388807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8ff59-92d1-469b-85de-62ba50bf240a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:04.389079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8ff59-92d1-469b-85de-62ba50bf240a admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:04.390026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f57517-3536-4d9d-ab28-74bdf46681de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:04.390141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8ff59-92d1-469b-85de-62ba50bf240a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:04.391957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8ff59-92d1-469b-85de-62ba50bf240a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:04.393417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f57517-3536-4d9d-ab28-74bdf46681de service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:04.416062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 684/1384] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:04.416062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f57517-3536-4d9d-ab28-74bdf46681de service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:04.417657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-84f57517-3536-4d9d-ab28-74bdf46681de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:04.443488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f57517-3536-4d9d-ab28-74bdf46681de service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:04.448547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:04.453858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:04.454927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:04.455633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:04.456025 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] PATH_INFO: `/v3/users/8f79f5fb6218464aa2bdef26aa5c21da` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:04.459554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f79f5fb6218464aa2bdef26aa5c21da)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:04.462156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:04.488125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 701/1385] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:04 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:04.489064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8ff59-92d1-469b-85de-62ba50bf240a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:04.493318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e0c8ff59-92d1-469b-85de-62ba50bf240a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:04.542317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f79f5fb6218464aa2bdef26aa5c21da'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:04.561787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f79f5fb6218464aa2bdef26aa5c21da'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:04.563845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f79f5fb6218464aa2bdef26aa5c21da'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:04.565954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f79f5fb6218464aa2bdef26aa5c21da'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:04.584505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f79f5fb6218464aa2bdef26aa5c21da'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:04.638372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 685/1386] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:04 2022] DELETE /identity/v3/users/8f79f5fb6218464aa2bdef26aa5c21da => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:04.639859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:04.643624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:04.683711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8ff59-92d1-469b-85de-62ba50bf240a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:04.734973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:04.797803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218a98a-027c-4202-a4d2-72a047738256 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:04.815183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1be4b2-b417-43b0-9033-93f230758ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:04.816178 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1be4b2-b417-43b0-9033-93f230758ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:04.820373 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1be4b2-b417-43b0-9033-93f230758ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:04.820373 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1be4b2-b417-43b0-9033-93f230758ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:04.820373 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1be4b2-b417-43b0-9033-93f230758ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:04.820373 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1be4b2-b417-43b0-9033-93f230758ae6 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:04.846649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:04.847890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:04.849860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:04.850530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] PATH_INFO: `/v3/projects/e5e217c805ce4bbc8d0d3cf8ae8e4568` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:04.851841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e217c805ce4bbc8d0d3cf8ae8e4568)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:04.854163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:04.868934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 686/1387] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:04 2022] POST /identity/v3/users => generated 282 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:04.873453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1be4b2-b417-43b0-9033-93f230758ae6 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:04.885040 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5e1be4b2-b417-43b0-9033-93f230758ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:04.921166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1be4b2-b417-43b0-9033-93f230758ae6 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:04.936131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:04.940623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:04.941100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:04.941500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] PATH_INFO: `/v3/users/66ed1729148f4417998c294d0aae2ee0` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:04.945692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ed1729148f4417998c294d0aae2ee0)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:04.949777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:04.962369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e217c805ce4bbc8d0d3cf8ae8e4568', 'request_id': 'req-0e365e01-13af-41a8-9341-124021c29867'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:04.973491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e217c805ce4bbc8d0d3cf8ae8e4568', 'request_id': 'req-0e365e01-13af-41a8-9341-124021c29867'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:04.990998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e217c805ce4bbc8d0d3cf8ae8e4568', 'request_id': 'req-0e365e01-13af-41a8-9341-124021c29867'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:05.020487 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] The token cache is being invalidate because project e5e217c805ce4bbc8d0d3cf8ae8e4568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:05.021032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:05.022596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e365e01-13af-41a8-9341-124021c29867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:05.024433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 702/1388] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:04 2022] DELETE /identity/v3/projects/e5e217c805ce4bbc8d0d3cf8ae8e4568 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:05.026350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d46c4-5bff-4204-a74b-81172a03bee5 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.027433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d46c4-5bff-4204-a74b-81172a03bee5 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.027943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d46c4-5bff-4204-a74b-81172a03bee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.054437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-986d46c4-5bff-4204-a74b-81172a03bee5 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:05.056155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-986d46c4-5bff-4204-a74b-81172a03bee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:05.061143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ed1729148f4417998c294d0aae2ee0'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:05.071122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ed1729148f4417998c294d0aae2ee0'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:05.073027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ed1729148f4417998c294d0aae2ee0'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:05.074442 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ed1729148f4417998c294d0aae2ee0'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:05.088406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ed1729148f4417998c294d0aae2ee0'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:05.107998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:05.132156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 687/1389] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:04 2022] DELETE /identity/v3/users/66ed1729148f4417998c294d0aae2ee0 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:05.138510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:05.142293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:05.202426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 703/1390] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:05.204121 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8c038-c919-416b-9029-26b16f5932ae None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.204667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8c038-c919-416b-9029-26b16f5932ae None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.206005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8c038-c919-416b-9029-26b16f5932ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.241426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1ed8c038-c919-416b-9029-26b16f5932ae None None] MFA Rules not processed for user `e0d4be0e0f744242a323ac9694aef9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:05.242958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1ed8c038-c919-416b-9029-26b16f5932ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:05.312264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee947d0-32d6-49e4-9067-182f87b98040 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:05.326000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4501ba-240a-488a-a481-684c550e0bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:05.326636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4501ba-240a-488a-a481-684c550e0bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.326919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4501ba-240a-488a-a481-684c550e0bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.327234 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4501ba-240a-488a-a481-684c550e0bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.328045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4501ba-240a-488a-a481-684c550e0bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:05.329873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4501ba-240a-488a-a481-684c550e0bb9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:05.354784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 688/1391] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:05.357380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87bc5-a7b3-4130-9302-2d53c39c4288 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.357672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87bc5-a7b3-4130-9302-2d53c39c4288 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.358586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87bc5-a7b3-4130-9302-2d53c39c4288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.414181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-a5a87bc5-a7b3-4130-9302-2d53c39c4288 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:05.422020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a5a87bc5-a7b3-4130-9302-2d53c39c4288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:05.452731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 704/1392] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:05.452731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0b1bf-e82d-45e2-b0be-a04f7c4054e7 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.452731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0b1bf-e82d-45e2-b0be-a04f7c4054e7 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.455183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 689/1393] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:05.457794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a87bc5-a7b3-4130-9302-2d53c39c4288 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:05.460193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0b1bf-e82d-45e2-b0be-a04f7c4054e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.462759 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a5a87bc5-a7b3-4130-9302-2d53c39c4288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:05.505156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-52d0b1bf-e82d-45e2-b0be-a04f7c4054e7 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:05.509372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-52d0b1bf-e82d-45e2-b0be-a04f7c4054e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:05.522853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:05.534670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 705/1394] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:05.534670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0b1bf-e82d-45e2-b0be-a04f7c4054e7 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:05.557184 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0b1bf-e82d-45e2-b0be-a04f7c4054e7 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:05.566045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f01b1e-ccc8-48b6-ac8c-06250bf47087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:05.566669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f01b1e-ccc8-48b6-ac8c-06250bf47087 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.566953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f01b1e-ccc8-48b6-ac8c-06250bf47087 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.567230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f01b1e-ccc8-48b6-ac8c-06250bf47087 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.567884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f01b1e-ccc8-48b6-ac8c-06250bf47087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:05.569689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f01b1e-ccc8-48b6-ac8c-06250bf47087 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:05.582446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 706/1395] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:05.583969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926669a4-3b9a-49c3-9794-b255a144e18a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.584431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926669a4-3b9a-49c3-9794-b255a144e18a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.584843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926669a4-3b9a-49c3-9794-b255a144e18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.594526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a87bc5-a7b3-4130-9302-2d53c39c4288 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:05.598579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934b374-5f35-4d54-813b-aa1c3045211f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:05.599504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b374-5f35-4d54-813b-aa1c3045211f service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.599788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b374-5f35-4d54-813b-aa1c3045211f service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.600067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b374-5f35-4d54-813b-aa1c3045211f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.614330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934b374-5f35-4d54-813b-aa1c3045211f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:05.615855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934b374-5f35-4d54-813b-aa1c3045211f service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:05.631966 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 690/1396] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:05.633532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0fa7b-ff53-42fc-a1ae-39d96a3c7d96 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.633918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0fa7b-ff53-42fc-a1ae-39d96a3c7d96 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.634290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0fa7b-ff53-42fc-a1ae-39d96a3c7d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.660710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-cbf0fa7b-ff53-42fc-a1ae-39d96a3c7d96 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:05.662307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cbf0fa7b-ff53-42fc-a1ae-39d96a3c7d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:05.671280 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-926669a4-3b9a-49c3-9794-b255a144e18a None None] MFA Rules not processed for user `9fc72f6965bb4814a5d85bd04769e4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:05.672924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-926669a4-3b9a-49c3-9794-b255a144e18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:05.683103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 691/1397] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:05.683103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0fa7b-ff53-42fc-a1ae-39d96a3c7d96 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:05.684805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cbf0fa7b-ff53-42fc-a1ae-39d96a3c7d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:05.837580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0fa7b-ff53-42fc-a1ae-39d96a3c7d96 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:05.855439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a00b01-efbe-4a65-b212-b9cf71fa0a2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:05.856078 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a00b01-efbe-4a65-b212-b9cf71fa0a2e service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.856398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a00b01-efbe-4a65-b212-b9cf71fa0a2e service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.856665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a00b01-efbe-4a65-b212-b9cf71fa0a2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.857048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 707/1398] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:05.860552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926669a4-3b9a-49c3-9794-b255a144e18a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:05.871016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a00b01-efbe-4a65-b212-b9cf71fa0a2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:05.872651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a00b01-efbe-4a65-b212-b9cf71fa0a2e service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:05.885144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926669a4-3b9a-49c3-9794-b255a144e18a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:05.894964 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 692/1399] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:05.895799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a00b01-efbe-4a65-b212-b9cf71fa0a2e service nova] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:05.897180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed2f85-1ae0-4b9f-9333-dd8447725ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:05.898014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed2f85-1ae0-4b9f-9333-dd8447725ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.898393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed2f85-1ae0-4b9f-9333-dd8447725ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.898786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed2f85-1ae0-4b9f-9333-dd8447725ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.921711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a00b01-efbe-4a65-b212-b9cf71fa0a2e service nova] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:05.926463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a93fc-ca72-4824-a82f-6927e00d8e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:05.927134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a93fc-ca72-4824-a82f-6927e00d8e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:05.927415 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a93fc-ca72-4824-a82f-6927e00d8e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:05.927688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a93fc-ca72-4824-a82f-6927e00d8e4d admin admin] PATH_INFO: `/v3/projects/ae7373c67edc493584fd03e50e54a8d0/users/f12e4ec3623540cdb5018a10a9c52337/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:05.931245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed2f85-1ae0-4b9f-9333-dd8447725ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:05.932930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed2f85-1ae0-4b9f-9333-dd8447725ec4 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:05.953223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 708/1400] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:05.954543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed2f85-1ae0-4b9f-9333-dd8447725ec4 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:05.958585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a93fc-ca72-4824-a82f-6927e00d8e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7373c67edc493584fd03e50e54a8d0, user_id=f12e4ec3623540cdb5018a10a9c52337, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:05.961515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a93fc-ca72-4824-a82f-6927e00d8e4d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:05.989652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 693/1401] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:05 2022] PUT /identity/v3/projects/ae7373c67edc493584fd03e50e54a8d0/users/f12e4ec3623540cdb5018a10a9c52337/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:05.991048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed2f85-1ae0-4b9f-9333-dd8447725ec4 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:06.001410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a93fc-ca72-4824-a82f-6927e00d8e4d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:06.003066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a38a93fc-ca72-4824-a82f-6927e00d8e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:06.035796 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:06.191761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a93fc-ca72-4824-a82f-6927e00d8e4d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:06.198939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339d6cd-bf7e-45a3-89c2-396e075f7b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:06.199596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339d6cd-bf7e-45a3-89c2-396e075f7b9a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:06.199877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339d6cd-bf7e-45a3-89c2-396e075f7b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:06.200151 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339d6cd-bf7e-45a3-89c2-396e075f7b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:06.200861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339d6cd-bf7e-45a3-89c2-396e075f7b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:06.201171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339d6cd-bf7e-45a3-89c2-396e075f7b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:06.204846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339d6cd-bf7e-45a3-89c2-396e075f7b9a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:06.206268 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e8f05-8fa7-4e1b-83b8-385d3d871d28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:06.206976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e8f05-8fa7-4e1b-83b8-385d3d871d28 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:06.207242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e8f05-8fa7-4e1b-83b8-385d3d871d28 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:06.207521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e8f05-8fa7-4e1b-83b8-385d3d871d28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:06.237746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:06.239261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 709/1402] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:06.241234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857ea64-fa16-41ed-8023-950eb6c483b8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:06.241529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857ea64-fa16-41ed-8023-950eb6c483b8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:06.242971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857ea64-fa16-41ed-8023-950eb6c483b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:06.306301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-b857ea64-fa16-41ed-8023-950eb6c483b8 None None] MFA Rules not processed for user `f12e4ec3623540cdb5018a10a9c52337`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:06.307715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b857ea64-fa16-41ed-8023-950eb6c483b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:06.342238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e8f05-8fa7-4e1b-83b8-385d3d871d28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:06.343894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e8f05-8fa7-4e1b-83b8-385d3d871d28 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:06.366668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 694/1403] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:06.367547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e8f05-8fa7-4e1b-83b8-385d3d871d28 service nova] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:06.368963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1c9e8f05-8fa7-4e1b-83b8-385d3d871d28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:06.489818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 710/1404] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:06.490360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857ea64-fa16-41ed-8023-950eb6c483b8 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:06.524634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857ea64-fa16-41ed-8023-950eb6c483b8 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:06.545621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e8f05-8fa7-4e1b-83b8-385d3d871d28 service nova] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:06.546959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e2757-b09e-4766-bc95-77662d0c58b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:06.547649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e2757-b09e-4766-bc95-77662d0c58b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:06.547927 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e2757-b09e-4766-bc95-77662d0c58b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:06.548265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e2757-b09e-4766-bc95-77662d0c58b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:06.549055 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e2757-b09e-4766-bc95-77662d0c58b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:06.552933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e2757-b09e-4766-bc95-77662d0c58b2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:06.557354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc18a78229064e8d8ea4321c3c23c4ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:06.558666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:06.558959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:06.559239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:06.561165 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:06.578441 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:06.580462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:06.599844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 695/1405] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:06.601013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:06.602386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:06.624057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821b4e0-36b0-46d8-8708-cf55c85fbb74 service placement] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:06.636865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:06.637621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:06.638524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:06.638935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] PATH_INFO: `/v3/projects/d2b475ae06da45b983011f02e8db23c0` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:06.642686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b475ae06da45b983011f02e8db23c0)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:06.644957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:06.651074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 711/1406] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:06 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:06.651074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e2757-b09e-4766-bc95-77662d0c58b2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:06.750531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b475ae06da45b983011f02e8db23c0', 'request_id': 'req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:06.763258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b475ae06da45b983011f02e8db23c0', 'request_id': 'req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:06.781173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b475ae06da45b983011f02e8db23c0', 'request_id': 'req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:06.805383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] The token cache is being invalidate because project d2b475ae06da45b983011f02e8db23c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:06.806746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:06.806746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:06.821560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 696/1407] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:06 2022] DELETE /identity/v3/projects/d2b475ae06da45b983011f02e8db23c0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:06.822068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:06.828993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:06.854933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e2757-b09e-4766-bc95-77662d0c58b2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:06.923457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:06.931673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:06.985890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944c5e-8696-415e-ae00-2dc3ce4b498d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:06.986911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944c5e-8696-415e-ae00-2dc3ce4b498d admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:06.987381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944c5e-8696-415e-ae00-2dc3ce4b498d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:06.987849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944c5e-8696-415e-ae00-2dc3ce4b498d admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:06.988819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2944c5e-8696-415e-ae00-2dc3ce4b498d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:06.990955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2944c5e-8696-415e-ae00-2dc3ce4b498d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:07.007997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9409c0-afa9-4eb4-8090-c2d48e7daf8e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:07.024065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc68d24-855c-47c8-a823-77d538250104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:07.026065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc68d24-855c-47c8-a823-77d538250104 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:07.026260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc68d24-855c-47c8-a823-77d538250104 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:07.026481 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc68d24-855c-47c8-a823-77d538250104 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:07.029542 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc68d24-855c-47c8-a823-77d538250104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:07.041432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc68d24-855c-47c8-a823-77d538250104 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:07.051835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 712/1408] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:06 2022] POST /identity/v3/projects => generated 363 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:07.060182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944c5e-8696-415e-ae00-2dc3ce4b498d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:07.066103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c2944c5e-8696-415e-ae00-2dc3ce4b498d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:07.113067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 697/1409] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:06 2022] POST /identity/v3/users => generated 290 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:07.114822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4f371-28f2-4d46-ab00-3ad12c6816fd None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:07.114822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4f371-28f2-4d46-ab00-3ad12c6816fd None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:07.115117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4f371-28f2-4d46-ab00-3ad12c6816fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:07.189203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-7ee4f371-28f2-4d46-ab00-3ad12c6816fd None None] MFA Rules not processed for user `4d74d7fd62cd464183f9df4ca581833c`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:07.190428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7ee4f371-28f2-4d46-ab00-3ad12c6816fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:07.248273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944c5e-8696-415e-ae00-2dc3ce4b498d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:07.268859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:07.271555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:07.271856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:07.273812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] PATH_INFO: `/v3/users/f8af325a9fac4d40b84c08bbb67c037e` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:07.282299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8af325a9fac4d40b84c08bbb67c037e)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:07.289934 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:07.440207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8af325a9fac4d40b84c08bbb67c037e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.456279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8af325a9fac4d40b84c08bbb67c037e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.457320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8af325a9fac4d40b84c08bbb67c037e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.459538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8af325a9fac4d40b84c08bbb67c037e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.481061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8af325a9fac4d40b84c08bbb67c037e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.488228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 698/1410] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:07.491844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4f371-28f2-4d46-ab00-3ad12c6816fd None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:07.495891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7ee4f371-28f2-4d46-ab00-3ad12c6816fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:07.570461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 713/1411] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:07 2022] DELETE /identity/v3/users/f8af325a9fac4d40b84c08bbb67c037e => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:07.573420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:07.574830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:07.581645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4f371-28f2-4d46-ab00-3ad12c6816fd None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:07.692450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:07.740962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:07.741901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:07.742392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:07.742808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] PATH_INFO: `/v3/projects/9ead5e2e710f4524b504763d5f809de1` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:07.745820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ead5e2e710f4524b504763d5f809de1)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:07.748654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:07.775262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b44d7-9585-4535-b9ad-0372bb194a90 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:07.791319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afdd71-6192-4ab9-b112-deda00ec674a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:07.792165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afdd71-6192-4ab9-b112-deda00ec674a admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:07.792589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afdd71-6192-4ab9-b112-deda00ec674a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:07.792958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afdd71-6192-4ab9-b112-deda00ec674a admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:07.793715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afdd71-6192-4ab9-b112-deda00ec674a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:07.800449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afdd71-6192-4ab9-b112-deda00ec674a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:07.861812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ead5e2e710f4524b504763d5f809de1', 'request_id': 'req-8b046a94-acff-4ed9-920e-c86207fd4b96'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.864175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 714/1412] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:07 2022] POST /identity/v3/users => generated 288 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:07.865018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:07.869320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afdd71-6192-4ab9-b112-deda00ec674a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:07.876756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e4afdd71-6192-4ab9-b112-deda00ec674a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:07.877972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ead5e2e710f4524b504763d5f809de1', 'request_id': 'req-8b046a94-acff-4ed9-920e-c86207fd4b96'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.895634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ead5e2e710f4524b504763d5f809de1', 'request_id': 'req-8b046a94-acff-4ed9-920e-c86207fd4b96'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.913104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afdd71-6192-4ab9-b112-deda00ec674a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:07.924522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] The token cache is being invalidate because project 9ead5e2e710f4524b504763d5f809de1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:07.926847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.928522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:07.937704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 699/1413] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:07 2022] DELETE /identity/v3/projects/9ead5e2e710f4524b504763d5f809de1 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:07.938908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:07.942756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:08.069437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06c2c5-6992-4576-8ccc-ebc504232804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:08.071153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06c2c5-6992-4576-8ccc-ebc504232804 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:08.071585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06c2c5-6992-4576-8ccc-ebc504232804 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:08.072004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06c2c5-6992-4576-8ccc-ebc504232804 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:08.072992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06c2c5-6992-4576-8ccc-ebc504232804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:08.075591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06c2c5-6992-4576-8ccc-ebc504232804 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:08.095470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 715/1414] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:08.100449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06c2c5-6992-4576-8ccc-ebc504232804 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:08.103426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9e06c2c5-6992-4576-8ccc-ebc504232804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:08.125947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06c2c5-6992-4576-8ccc-ebc504232804 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:08.134846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b046a94-acff-4ed9-920e-c86207fd4b96 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:08.140166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffad936-4fe3-4c62-bff9-1ab636eafc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:08.140739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffad936-4fe3-4c62-bff9-1ab636eafc99 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:08.141176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffad936-4fe3-4c62-bff9-1ab636eafc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:08.141531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffad936-4fe3-4c62-bff9-1ab636eafc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:08.142124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffad936-4fe3-4c62-bff9-1ab636eafc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:08.143865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffad936-4fe3-4c62-bff9-1ab636eafc99 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:08.153474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04faade-163f-42db-a38c-2393760030ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:08.154167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04faade-163f-42db-a38c-2393760030ba service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:08.154517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04faade-163f-42db-a38c-2393760030ba service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:08.154806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04faade-163f-42db-a38c-2393760030ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:08.159178 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c04faade-163f-42db-a38c-2393760030ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:08.174001 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 716/1415] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:08.177333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffad936-4fe3-4c62-bff9-1ab636eafc99 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:08.180837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5ffad936-4fe3-4c62-bff9-1ab636eafc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:08.217554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffad936-4fe3-4c62-bff9-1ab636eafc99 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:08.226240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:08.227115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:08.227418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:08.227685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] PATH_INFO: `/v3/users/aa5ecc0bd4de48b3876ea9657065187e` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:08.229915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5ecc0bd4de48b3876ea9657065187e)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:08.232439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:08.342228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04faade-163f-42db-a38c-2393760030ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:08.346037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04faade-163f-42db-a38c-2393760030ba service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:08.373490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 700/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:08.375802 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04faade-163f-42db-a38c-2393760030ba service nova] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:08.378623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c04faade-163f-42db-a38c-2393760030ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:08.381666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5ecc0bd4de48b3876ea9657065187e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:08.406926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5ecc0bd4de48b3876ea9657065187e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:08.409384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5ecc0bd4de48b3876ea9657065187e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:08.412084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5ecc0bd4de48b3876ea9657065187e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:08.423252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04faade-163f-42db-a38c-2393760030ba service nova] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:08.430429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5ecc0bd4de48b3876ea9657065187e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:08.434687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528e5e8-8467-4fc5-a992-34ee59ffbc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:08.441802 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528e5e8-8467-4fc5-a992-34ee59ffbc66 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:08.443281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528e5e8-8467-4fc5-a992-34ee59ffbc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:08.443677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528e5e8-8467-4fc5-a992-34ee59ffbc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:08.445686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528e5e8-8467-4fc5-a992-34ee59ffbc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:08.449195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528e5e8-8467-4fc5-a992-34ee59ffbc66 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:08.471206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 701/1417] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:08.475235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528e5e8-8467-4fc5-a992-34ee59ffbc66 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:08.478166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c528e5e8-8467-4fc5-a992-34ee59ffbc66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:08.492403 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 717/1418] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:08 2022] DELETE /identity/v3/users/aa5ecc0bd4de48b3876ea9657065187e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:08.493043 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:08.508891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:08.549050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528e5e8-8467-4fc5-a992-34ee59ffbc66 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:08.589933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:08.698239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7c97e-53f0-4809-9429-df0ec6efdf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:08.702259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7c97e-53f0-4809-9429-df0ec6efdf1b admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:08.704132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7c97e-53f0-4809-9429-df0ec6efdf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:08.704647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7c97e-53f0-4809-9429-df0ec6efdf1b admin admin] PATH_INFO: `/v3/projects/b0914029e60343d8a4921a3e3cf52067/users/a7377262284240d3ad4e3f34209ba360/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:08.717384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104a68f-2d56-4a23-8107-7ca38a6cc1e2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:08.724597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:08.734663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693fa46-6740-42c1-b9bc-87e837bd3d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:08.735353 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693fa46-6740-42c1-b9bc-87e837bd3d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:08.736681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693fa46-6740-42c1-b9bc-87e837bd3d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:08.736968 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693fa46-6740-42c1-b9bc-87e837bd3d53 admin admin] PATH_INFO: `/v3/projects/0b80f77da11e430da8f615d4bbf74063/users/b12517a82e67441ba87a79adc7c2ee93/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:08.738506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7c97e-53f0-4809-9429-df0ec6efdf1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0914029e60343d8a4921a3e3cf52067, user_id=a7377262284240d3ad4e3f34209ba360, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:08.742182 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7c97e-53f0-4809-9429-df0ec6efdf1b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:08.775327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 702/1419] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:08 2022] PUT /identity/v3/projects/b0914029e60343d8a4921a3e3cf52067/users/a7377262284240d3ad4e3f34209ba360/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:08.778330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7c97e-53f0-4809-9429-df0ec6efdf1b admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:08.784681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693fa46-6740-42c1-b9bc-87e837bd3d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b80f77da11e430da8f615d4bbf74063, user_id=b12517a82e67441ba87a79adc7c2ee93, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:08.790671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693fa46-6740-42c1-b9bc-87e837bd3d53 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:08.822425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 718/1420] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:08 2022] PUT /identity/v3/projects/0b80f77da11e430da8f615d4bbf74063/users/b12517a82e67441ba87a79adc7c2ee93/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:08.837022 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49f6e-3f63-4b6d-a666-c72737f6491c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:08.837571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49f6e-3f63-4b6d-a666-c72737f6491c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:08.837954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49f6e-3f63-4b6d-a666-c72737f6491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:08.912897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-61c49f6e-3f63-4b6d-a666-c72737f6491c None None] MFA Rules not processed for user `b12517a82e67441ba87a79adc7c2ee93`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:08.916368 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-61c49f6e-3f63-4b6d-a666-c72737f6491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:08.957687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7c97e-53f0-4809-9429-df0ec6efdf1b admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:09.065958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef7a6-bd85-4db3-a720-42f484cfa7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:09.069820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef7a6-bd85-4db3-a720-42f484cfa7cf admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:09.070393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef7a6-bd85-4db3-a720-42f484cfa7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:09.070792 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 719/1421] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:41:09.072287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef7a6-bd85-4db3-a720-42f484cfa7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:09.073033 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cef7a6-bd85-4db3-a720-42f484cfa7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:09.075511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cef7a6-bd85-4db3-a720-42f484cfa7cf admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:09.076456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c49f6e-3f63-4b6d-a666-c72737f6491c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:09.088729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 703/1422] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:09.093504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef7a6-bd85-4db3-a720-42f484cfa7cf admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:09.100506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c49f6e-3f63-4b6d-a666-c72737f6491c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:09.110385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5390ea-baa0-419f-b03d-67fcf20f2e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:09.111505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5390ea-baa0-419f-b03d-67fcf20f2e24 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:09.111902 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef7a6-bd85-4db3-a720-42f484cfa7cf admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:09.112517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5390ea-baa0-419f-b03d-67fcf20f2e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:09.112947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5390ea-baa0-419f-b03d-67fcf20f2e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:09.114185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5390ea-baa0-419f-b03d-67fcf20f2e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:09.118485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a05f6d-8414-4870-bc19-591e6db16261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:09.119030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5390ea-baa0-419f-b03d-67fcf20f2e24 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:09.119431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a05f6d-8414-4870-bc19-591e6db16261 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:09.119845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a05f6d-8414-4870-bc19-591e6db16261 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:09.120229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a05f6d-8414-4870-bc19-591e6db16261 admin admin] PATH_INFO: `/v3/projects/b0914029e60343d8a4921a3e3cf52067/users/a7377262284240d3ad4e3f34209ba360/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:09.123162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a05f6d-8414-4870-bc19-591e6db16261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0914029e60343d8a4921a3e3cf52067, user_id=a7377262284240d3ad4e3f34209ba360, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:09.126499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a05f6d-8414-4870-bc19-591e6db16261 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:09.160574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 704/1423] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:09 2022] PUT /identity/v3/projects/b0914029e60343d8a4921a3e3cf52067/users/a7377262284240d3ad4e3f34209ba360/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:09.177428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1c6be-50ee-45de-9ab5-e77c7aaf93c3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:09.177428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1c6be-50ee-45de-9ab5-e77c7aaf93c3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:09.177428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1c6be-50ee-45de-9ab5-e77c7aaf93c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:09.179080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 720/1424] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:09 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:09.187944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5390ea-baa0-419f-b03d-67fcf20f2e24 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:09.238240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-51f1c6be-50ee-45de-9ab5-e77c7aaf93c3 None None] MFA Rules not processed for user `a7377262284240d3ad4e3f34209ba360`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:09.239553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-51f1c6be-50ee-45de-9ab5-e77c7aaf93c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:09.316396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5390ea-baa0-419f-b03d-67fcf20f2e24 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:09.388677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbea47b-e34e-48aa-9bbc-f2bef12690a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:09.389594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbea47b-e34e-48aa-9bbc-f2bef12690a9 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:09.390025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbea47b-e34e-48aa-9bbc-f2bef12690a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:09.390716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbea47b-e34e-48aa-9bbc-f2bef12690a9 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:09.391850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbea47b-e34e-48aa-9bbc-f2bef12690a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:09.409013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbea47b-e34e-48aa-9bbc-f2bef12690a9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:09.440013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:09.453115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 721/1425] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:09 2022] POST /identity/v3/users => generated 296 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:09.454624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbea47b-e34e-48aa-9bbc-f2bef12690a9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:09.457960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-dfbea47b-e34e-48aa-9bbc-f2bef12690a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:09.491235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbea47b-e34e-48aa-9bbc-f2bef12690a9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:09.505015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:09.505970 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:09.506068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:09.506587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] PATH_INFO: `/v3/projects/be95a4f8cd6441e380dd6f303027cf04` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:09.507595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be95a4f8cd6441e380dd6f303027cf04)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:09.509572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:09.604070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 705/1426] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:09.606007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1c6be-50ee-45de-9ab5-e77c7aaf93c3 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:09.619791 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:09.631315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1c6be-50ee-45de-9ab5-e77c7aaf93c3 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:09.649138 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f32a6-4b09-4c04-8cb9-8683ec06b63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:09.649896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f32a6-4b09-4c04-8cb9-8683ec06b63e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:09.650067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f32a6-4b09-4c04-8cb9-8683ec06b63e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:09.650311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f32a6-4b09-4c04-8cb9-8683ec06b63e admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:09.651046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f32a6-4b09-4c04-8cb9-8683ec06b63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:09.653350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f32a6-4b09-4c04-8cb9-8683ec06b63e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:09.680767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 706/1427] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:09.682060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f32a6-4b09-4c04-8cb9-8683ec06b63e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:09.685225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6a1f32a6-4b09-4c04-8cb9-8683ec06b63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:09.743758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be95a4f8cd6441e380dd6f303027cf04', 'request_id': 'req-791b04e5-4dcc-4998-a565-1772a4b39670'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:09.766806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be95a4f8cd6441e380dd6f303027cf04', 'request_id': 'req-791b04e5-4dcc-4998-a565-1772a4b39670'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:09.786211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be95a4f8cd6441e380dd6f303027cf04', 'request_id': 'req-791b04e5-4dcc-4998-a565-1772a4b39670'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:09.822260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] The token cache is being invalidate because project be95a4f8cd6441e380dd6f303027cf04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:09.822575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:09.826304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:09.828844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 722/1428] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:09 2022] DELETE /identity/v3/projects/be95a4f8cd6441e380dd6f303027cf04 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:09.831465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:09.834547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:09.871137 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f32a6-4b09-4c04-8cb9-8683ec06b63e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:10.031303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:10.032240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:10.032515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:10.032937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:10.040083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:10.057055 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b04e5-4dcc-4998-a565-1772a4b39670 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:10.069763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f836749-52f1-431d-9cbf-dacfac36b975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:10.070695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f836749-52f1-431d-9cbf-dacfac36b975 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:10.071131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f836749-52f1-431d-9cbf-dacfac36b975 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:10.072084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f836749-52f1-431d-9cbf-dacfac36b975 admin admin] PATH_INFO: `/v3/projects/f10f0aa4929d465cbe5dbe6807f6f080/users/7f07a194efc14c9eb56eb25f7c8c965a/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:10.126232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f836749-52f1-431d-9cbf-dacfac36b975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10f0aa4929d465cbe5dbe6807f6f080, user_id=7f07a194efc14c9eb56eb25f7c8c965a, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:10.131339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f836749-52f1-431d-9cbf-dacfac36b975 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:10.169978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 723/1429] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:09 2022] PUT /identity/v3/projects/f10f0aa4929d465cbe5dbe6807f6f080/users/7f07a194efc14c9eb56eb25f7c8c965a/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:10.172601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e084581-969d-4fb0-b40f-1c5b2b410c2a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:10.172947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e084581-969d-4fb0-b40f-1c5b2b410c2a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:10.173377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e084581-969d-4fb0-b40f-1c5b2b410c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:10.255744 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:10.257387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:10.277168 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-7e084581-969d-4fb0-b40f-1c5b2b410c2a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:10.282609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7e084581-969d-4fb0-b40f-1c5b2b410c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:10.298039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 707/1430] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:10.299187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:10.303953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:10.346303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66280fb-4593-4725-bea6-1c02eb23f710 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:10.489303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 724/1431] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:10.490461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e084581-969d-4fb0-b40f-1c5b2b410c2a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:10.497984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:10.518355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:10.519260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:10.520895 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:10.521322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] PATH_INFO: `/v3/projects/d867bf225fff4c568bac88091cbb129b` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:10.525645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d867bf225fff4c568bac88091cbb129b)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:10.531847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:10.541932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e084581-969d-4fb0-b40f-1c5b2b410c2a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:10.565442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b88006-c671-42e7-b886-909d7aba22ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:10.566036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b88006-c671-42e7-b886-909d7aba22ba admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:10.566327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b88006-c671-42e7-b886-909d7aba22ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:10.571409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b88006-c671-42e7-b886-909d7aba22ba admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:10.577324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b88006-c671-42e7-b886-909d7aba22ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:10.582961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b88006-c671-42e7-b886-909d7aba22ba admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:10.588632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:10.607822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 725/1432] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:10.609832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505c48c-b028-45d9-b1ec-7f4853b90fb6 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:10.610144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505c48c-b028-45d9-b1ec-7f4853b90fb6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:10.616570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505c48c-b028-45d9-b1ec-7f4853b90fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:10.695906 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd867bf225fff4c568bac88091cbb129b', 'request_id': 'req-3dea8179-e911-43b6-b7f2-fa8bff991a76'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:10.732963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd867bf225fff4c568bac88091cbb129b', 'request_id': 'req-3dea8179-e911-43b6-b7f2-fa8bff991a76'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:10.748753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d505c48c-b028-45d9-b1ec-7f4853b90fb6 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:10.750861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d505c48c-b028-45d9-b1ec-7f4853b90fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:10.762079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd867bf225fff4c568bac88091cbb129b', 'request_id': 'req-3dea8179-e911-43b6-b7f2-fa8bff991a76'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:10.789395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] The token cache is being invalidate because project d867bf225fff4c568bac88091cbb129b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:10.789841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:10.790625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3dea8179-e911-43b6-b7f2-fa8bff991a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:10.792998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 708/1433] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:10 2022] DELETE /identity/v3/projects/d867bf225fff4c568bac88091cbb129b => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:10.794410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb476491-bea5-4634-b82d-a72a2d6c6a6a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:10.796657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb476491-bea5-4634-b82d-a72a2d6c6a6a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:10.797043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb476491-bea5-4634-b82d-a72a2d6c6a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:10.874198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-bb476491-bea5-4634-b82d-a72a2d6c6a6a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:10.879392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bb476491-bea5-4634-b82d-a72a2d6c6a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:10.937147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 726/1434] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:10.938624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505c48c-b028-45d9-b1ec-7f4853b90fb6 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:10.940399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d505c48c-b028-45d9-b1ec-7f4853b90fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:11.070115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 709/1435] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:11.072290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb476491-bea5-4634-b82d-a72a2d6c6a6a None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:11.080087 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bb476491-bea5-4634-b82d-a72a2d6c6a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:11.130153 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb476491-bea5-4634-b82d-a72a2d6c6a6a None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:11.153202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96f36-db3a-4211-9afd-0d3f0a12bf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:11.155977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96f36-db3a-4211-9afd-0d3f0a12bf66 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:11.156265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96f36-db3a-4211-9afd-0d3f0a12bf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:11.161344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96f36-db3a-4211-9afd-0d3f0a12bf66 admin admin] PATH_INFO: `/v3/projects/f10f0aa4929d465cbe5dbe6807f6f080/users/7f07a194efc14c9eb56eb25f7c8c965a/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:11.165411 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505c48c-b028-45d9-b1ec-7f4853b90fb6 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:11.166394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96f36-db3a-4211-9afd-0d3f0a12bf66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10f0aa4929d465cbe5dbe6807f6f080, user_id=7f07a194efc14c9eb56eb25f7c8c965a, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:11.175595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96f36-db3a-4211-9afd-0d3f0a12bf66 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:11.185600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b89e-18d6-4701-a014-e0ff5de84299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:11.186267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b89e-18d6-4701-a014-e0ff5de84299 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:11.186548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b89e-18d6-4701-a014-e0ff5de84299 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:11.186823 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b89e-18d6-4701-a014-e0ff5de84299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:11.191362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a9e4b89e-18d6-4701-a014-e0ff5de84299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:11.224154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 710/1436] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:11 2022] PUT /identity/v3/projects/f10f0aa4929d465cbe5dbe6807f6f080/users/7f07a194efc14c9eb56eb25f7c8c965a/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:11.230832 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c34a9-1a98-4f34-8b9c-76327feb7964 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:11.231213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c34a9-1a98-4f34-8b9c-76327feb7964 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:11.231541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c34a9-1a98-4f34-8b9c-76327feb7964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:11.300489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-740c34a9-1a98-4f34-8b9c-76327feb7964 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:11.301948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-740c34a9-1a98-4f34-8b9c-76327feb7964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:11.377600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b89e-18d6-4701-a014-e0ff5de84299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:11.379399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b89e-18d6-4701-a014-e0ff5de84299 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:11.390849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 727/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:11.391819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b89e-18d6-4701-a014-e0ff5de84299 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:11.411951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b89e-18d6-4701-a014-e0ff5de84299 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:11.457995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:11.481685 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 711/1438] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:11.481685 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddae4a-4f59-42a2-b62d-349cb41b7256 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:11.481685 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddae4a-4f59-42a2-b62d-349cb41b7256 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:11.481685 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddae4a-4f59-42a2-b62d-349cb41b7256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:11.552491 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-58ddae4a-4f59-42a2-b62d-349cb41b7256 None None] MFA Rules not processed for user `7f07a194efc14c9eb56eb25f7c8c965a`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:11.556520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-58ddae4a-4f59-42a2-b62d-349cb41b7256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:11.557441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:11.560920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:11.562872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:11.563166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:11.564306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:11.564651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:11.571001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:11.572550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:11.608381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 728/1439] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:11.609417 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:11.615948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:11.775646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 712/1440] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:11.777649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d5643-ffb3-4e2b-a61b-e611012d3317 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:11.778003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d5643-ffb3-4e2b-a61b-e611012d3317 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:11.778427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d5643-ffb3-4e2b-a61b-e611012d3317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:11.818939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-525d5643-ffb3-4e2b-a61b-e611012d3317 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:11.821248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-525d5643-ffb3-4e2b-a61b-e611012d3317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:11.835046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 713/1441] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:11.836233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d5643-ffb3-4e2b-a61b-e611012d3317 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:11.855235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d5643-ffb3-4e2b-a61b-e611012d3317 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:11.869885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb153f-71aa-42b5-bbd1-84096fd53358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:11.873341 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb153f-71aa-42b5-bbd1-84096fd53358 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:11.873341 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb153f-71aa-42b5-bbd1-84096fd53358 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:11.873759 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb153f-71aa-42b5-bbd1-84096fd53358 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:11.875420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb153f-71aa-42b5-bbd1-84096fd53358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:11.880596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb153f-71aa-42b5-bbd1-84096fd53358 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:11.919720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea480c-6468-48f2-b1b3-7317638b0b26 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:11.947107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 714/1442] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:11 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:11.950634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3e757-6c80-4e6d-a186-754df5d12bdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:11.952361 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb153f-71aa-42b5-bbd1-84096fd53358 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:11.952546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3e757-6c80-4e6d-a186-754df5d12bdc service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:11.958328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3e757-6c80-4e6d-a186-754df5d12bdc service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:11.958754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3e757-6c80-4e6d-a186-754df5d12bdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:12.118745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb153f-71aa-42b5-bbd1-84096fd53358 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:12.126122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad7508-bb2c-4789-9d2c-1b21a55c0e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:12.126934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad7508-bb2c-4789-9d2c-1b21a55c0e46 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:12.127223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad7508-bb2c-4789-9d2c-1b21a55c0e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:12.127496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad7508-bb2c-4789-9d2c-1b21a55c0e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:12.137774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d3e757-6c80-4e6d-a186-754df5d12bdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:12.139665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d3e757-6c80-4e6d-a186-754df5d12bdc service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:12.166761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 729/1443] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:41:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:41:12.168338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3e757-6c80-4e6d-a186-754df5d12bdc service swift] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:12.200086 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3e757-6c80-4e6d-a186-754df5d12bdc service swift] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:12.283514 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:12.285740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad7508-bb2c-4789-9d2c-1b21a55c0e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:12.287372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad7508-bb2c-4789-9d2c-1b21a55c0e46 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:12.336157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 715/1444] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:12.343308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad7508-bb2c-4789-9d2c-1b21a55c0e46 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:12.351593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ccce3-7bf0-4997-bba5-127e092903ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:12.352458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ccce3-7bf0-4997-bba5-127e092903ff admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:12.352884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ccce3-7bf0-4997-bba5-127e092903ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:12.356686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ccce3-7bf0-4997-bba5-127e092903ff admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:12.360499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ccce3-7bf0-4997-bba5-127e092903ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:12.361243 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ccce3-7bf0-4997-bba5-127e092903ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:12.364072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ccce3-7bf0-4997-bba5-127e092903ff admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:12.378747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad7508-bb2c-4789-9d2c-1b21a55c0e46 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:12.396708 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e568a65-84ff-4823-b7f1-0b6bfa09ae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:12.397468 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e568a65-84ff-4823-b7f1-0b6bfa09ae51 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:12.397755 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e568a65-84ff-4823-b7f1-0b6bfa09ae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:12.398033 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e568a65-84ff-4823-b7f1-0b6bfa09ae51 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:12.410600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e568a65-84ff-4823-b7f1-0b6bfa09ae51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:12.412823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e568a65-84ff-4823-b7f1-0b6bfa09ae51 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:12.451404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 730/1445] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:12.452582 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ccce3-7bf0-4997-bba5-127e092903ff admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:12.484876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ccce3-7bf0-4997-bba5-127e092903ff admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:12.494357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 716/1446] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:12 2022] POST /identity/v3/users => generated 284 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:12.495569 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e568a65-84ff-4823-b7f1-0b6bfa09ae51 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:12.497432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a9e3-1280-40ed-b643-645a204b6fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:12.498402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a9e3-1280-40ed-b643-645a204b6fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:12.498891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a9e3-1280-40ed-b643-645a204b6fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:12.499579 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a9e3-1280-40ed-b643-645a204b6fa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:12.500573 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a9e3-1280-40ed-b643-645a204b6fa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:12.506222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a9e3-1280-40ed-b643-645a204b6fa9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:12.530952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:12.543499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e568a65-84ff-4823-b7f1-0b6bfa09ae51 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:12.565493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f892b19-e7d0-40b6-9db6-b125bf3876a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:12.566562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f892b19-e7d0-40b6-9db6-b125bf3876a8 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:12.566562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f892b19-e7d0-40b6-9db6-b125bf3876a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:12.567224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f892b19-e7d0-40b6-9db6-b125bf3876a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:12.567405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f892b19-e7d0-40b6-9db6-b125bf3876a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:12.569175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f892b19-e7d0-40b6-9db6-b125bf3876a8 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:12.610138 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 717/1447] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:12.617133 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f892b19-e7d0-40b6-9db6-b125bf3876a8 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:12.653836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f892b19-e7d0-40b6-9db6-b125bf3876a8 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:12.668924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56754ebc-2cf3-427b-9ed6-3188473f9dd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:12.669565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56754ebc-2cf3-427b-9ed6-3188473f9dd2 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:12.669845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56754ebc-2cf3-427b-9ed6-3188473f9dd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:12.670118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56754ebc-2cf3-427b-9ed6-3188473f9dd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:12.684101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-56754ebc-2cf3-427b-9ed6-3188473f9dd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:12.793289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 731/1448] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:12 2022] POST /identity/v3/projects => generated 385 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:12.795245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a9e3-1280-40ed-b643-645a204b6fa9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:12.973277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56754ebc-2cf3-427b-9ed6-3188473f9dd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:12.974763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56754ebc-2cf3-427b-9ed6-3188473f9dd2 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:12.998672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a9e3-1280-40ed-b643-645a204b6fa9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:13.013173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 718/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:13.014767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56754ebc-2cf3-427b-9ed6-3188473f9dd2 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:13.065349 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56754ebc-2cf3-427b-9ed6-3188473f9dd2 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:13.065554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45ffbf-7cd0-4ef6-8138-0c0c966993cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:13.069110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45ffbf-7cd0-4ef6-8138-0c0c966993cd admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:13.071061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45ffbf-7cd0-4ef6-8138-0c0c966993cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:13.071577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45ffbf-7cd0-4ef6-8138-0c0c966993cd admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:13.078132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45ffbf-7cd0-4ef6-8138-0c0c966993cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:13.083278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45ffbf-7cd0-4ef6-8138-0c0c966993cd admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:13.084159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2684eb-8818-48dd-83e7-ebd32c6078a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:13.087087 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2684eb-8818-48dd-83e7-ebd32c6078a0 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:13.091691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2684eb-8818-48dd-83e7-ebd32c6078a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:13.091989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2684eb-8818-48dd-83e7-ebd32c6078a0 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:13.092716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2684eb-8818-48dd-83e7-ebd32c6078a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:13.103571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2684eb-8818-48dd-83e7-ebd32c6078a0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:13.137540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 732/1450] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:13.141009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45ffbf-7cd0-4ef6-8138-0c0c966993cd admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:13.173694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45ffbf-7cd0-4ef6-8138-0c0c966993cd admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:13.204542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669bdca-b388-4f54-bbc7-f68e4a927dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:13.205225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669bdca-b388-4f54-bbc7-f68e4a927dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:13.209922 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669bdca-b388-4f54-bbc7-f68e4a927dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:13.213794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669bdca-b388-4f54-bbc7-f68e4a927dcd admin admin] PATH_INFO: `/v3/projects/5123134e4b3d4fa2a28b1fa77c7a403c/users/fd3629e7e9ae43d38f4f1d1bbd76024b/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:13.224757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 719/1451] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:13 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:13.229390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2684eb-8818-48dd-83e7-ebd32c6078a0 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:13.283059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2684eb-8818-48dd-83e7-ebd32c6078a0 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:13.293316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669bdca-b388-4f54-bbc7-f68e4a927dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5123134e4b3d4fa2a28b1fa77c7a403c, user_id=fd3629e7e9ae43d38f4f1d1bbd76024b, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:13.304618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669bdca-b388-4f54-bbc7-f68e4a927dcd admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:13.319787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195524bf-b038-4fb5-9c6c-2ebe485b99a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:13.325296 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195524bf-b038-4fb5-9c6c-2ebe485b99a6 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:13.325649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195524bf-b038-4fb5-9c6c-2ebe485b99a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:13.325947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195524bf-b038-4fb5-9c6c-2ebe485b99a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:13.326676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195524bf-b038-4fb5-9c6c-2ebe485b99a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:13.331048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195524bf-b038-4fb5-9c6c-2ebe485b99a6 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:13.365818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 733/1452] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:13 2022] PUT /identity/v3/projects/5123134e4b3d4fa2a28b1fa77c7a403c/users/fd3629e7e9ae43d38f4f1d1bbd76024b/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:13.381901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57f013-5bd3-46bf-8b3e-f09b38d22656 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:13.382188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57f013-5bd3-46bf-8b3e-f09b38d22656 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:13.382468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57f013-5bd3-46bf-8b3e-f09b38d22656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:13.417979 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 720/1453] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:13.422401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195524bf-b038-4fb5-9c6c-2ebe485b99a6 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:13.466693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195524bf-b038-4fb5-9c6c-2ebe485b99a6 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:13.584870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9e57f013-5bd3-46bf-8b3e-f09b38d22656 None None] MFA Rules not processed for user `fd3629e7e9ae43d38f4f1d1bbd76024b`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:13.601599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9e57f013-5bd3-46bf-8b3e-f09b38d22656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:13.719955 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a635af-2168-465e-aba4-156ed51e2c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:13.721324 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a635af-2168-465e-aba4-156ed51e2c89 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:13.721731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a635af-2168-465e-aba4-156ed51e2c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:13.722101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a635af-2168-465e-aba4-156ed51e2c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:13.722874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a635af-2168-465e-aba4-156ed51e2c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:13.725108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a635af-2168-465e-aba4-156ed51e2c89 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:13.766583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 721/1454] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:13.766583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99106a-1a30-44de-8772-e02484fb005f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:13.766583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99106a-1a30-44de-8772-e02484fb005f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:13.767170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99106a-1a30-44de-8772-e02484fb005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:13.845084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-cc99106a-1a30-44de-8772-e02484fb005f None None] MFA Rules not processed for user `ae4b023dbf114bf98d0e38ea7fec94fa`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:13.850436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cc99106a-1a30-44de-8772-e02484fb005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:13.852204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 734/1455] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:41:13.853204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57f013-5bd3-46bf-8b3e-f09b38d22656 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:13.875003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57f013-5bd3-46bf-8b3e-f09b38d22656 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:13.908160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144231c9-aa2c-4b02-a319-253aac262b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:13.908160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144231c9-aa2c-4b02-a319-253aac262b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:13.908160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144231c9-aa2c-4b02-a319-253aac262b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:13.908160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144231c9-aa2c-4b02-a319-253aac262b2f admin admin] PATH_INFO: `/v3/projects/27fa1a3cd3504709a34b76f876d21b43/users/0a99208281174f94b274c11caeb376e2/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:13.950608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:13.981289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144231c9-aa2c-4b02-a319-253aac262b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=27fa1a3cd3504709a34b76f876d21b43, user_id=0a99208281174f94b274c11caeb376e2, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:13.984178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144231c9-aa2c-4b02-a319-253aac262b2f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:14.025051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 722/1456] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Oct 20 14:41:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:14.025258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 735/1457] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:13 2022] PUT /identity/v3/projects/27fa1a3cd3504709a34b76f876d21b43/users/0a99208281174f94b274c11caeb376e2/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:14.025361 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99106a-1a30-44de-8772-e02484fb005f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:14.026684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1b5c2-3eb6-417b-801d-a702ae64508b None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:14.026892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1b5c2-3eb6-417b-801d-a702ae64508b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:14.027250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1b5c2-3eb6-417b-801d-a702ae64508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:14.066765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99106a-1a30-44de-8772-e02484fb005f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:14.088485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:14.138761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-84c1b5c2-3eb6-417b-801d-a702ae64508b None None] MFA Rules not processed for user `e0d4be0e0f744242a323ac9694aef9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:14.140962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-84c1b5c2-3eb6-417b-801d-a702ae64508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:14.270321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d94d6e-74dc-406b-a5ed-7c7275f927c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:14.270958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d94d6e-74dc-406b-a5ed-7c7275f927c8 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:14.273370 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d94d6e-74dc-406b-a5ed-7c7275f927c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:14.273719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d94d6e-74dc-406b-a5ed-7c7275f927c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:14.274877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d94d6e-74dc-406b-a5ed-7c7275f927c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:14.281267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d94d6e-74dc-406b-a5ed-7c7275f927c8 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:14.379279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 723/1458] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:14 2022] POST /identity/v3/projects => generated 369 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:14.381954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1d1b5-6cfe-4edb-a63d-fc61ecd1679c None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:14.383372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1d1b5-6cfe-4edb-a63d-fc61ecd1679c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:14.383665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1d1b5-6cfe-4edb-a63d-fc61ecd1679c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:14.392818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 736/1459] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:14.394453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1b5c2-3eb6-417b-801d-a702ae64508b None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:14.399197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-84c1b5c2-3eb6-417b-801d-a702ae64508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:14.499120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ccb1d1b5-6cfe-4edb-a63d-fc61ecd1679c None None] MFA Rules not processed for user `0a99208281174f94b274c11caeb376e2`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:14.500372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ccb1d1b5-6cfe-4edb-a63d-fc61ecd1679c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:14.542899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1b5c2-3eb6-417b-801d-a702ae64508b None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:14.690593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 724/1460] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:14.693443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3357a4-0cfa-4b14-96b7-485bf391c656 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:14.693723 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3357a4-0cfa-4b14-96b7-485bf391c656 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:14.694008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3357a4-0cfa-4b14-96b7-485bf391c656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:14.699874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131a2dd-090c-4ebf-b9fd-c81223667c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:14.700552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131a2dd-090c-4ebf-b9fd-c81223667c03 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:14.700991 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131a2dd-090c-4ebf-b9fd-c81223667c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:14.701279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131a2dd-090c-4ebf-b9fd-c81223667c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:14.717923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131a2dd-090c-4ebf-b9fd-c81223667c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:14.726066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131a2dd-090c-4ebf-b9fd-c81223667c03 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:14.749109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-2a3357a4-0cfa-4b14-96b7-485bf391c656 None None] MFA Rules not processed for user `9fc72f6965bb4814a5d85bd04769e4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:14.749109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2a3357a4-0cfa-4b14-96b7-485bf391c656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:14.752684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 737/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:14.754048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131a2dd-090c-4ebf-b9fd-c81223667c03 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:14.928007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:14.930409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 725/1462] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:14.931975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3357a4-0cfa-4b14-96b7-485bf391c656 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:14.965122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131a2dd-090c-4ebf-b9fd-c81223667c03 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:14.982471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec475cf-e0da-4f53-976c-44234bc65f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:14.983257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec475cf-e0da-4f53-976c-44234bc65f7b admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:14.983605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec475cf-e0da-4f53-976c-44234bc65f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:14.985136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec475cf-e0da-4f53-976c-44234bc65f7b admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:14.989634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec475cf-e0da-4f53-976c-44234bc65f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:14.992760 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec475cf-e0da-4f53-976c-44234bc65f7b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:15.060173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 738/1463] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:14 2022] POST /identity/v3/users => generated 291 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:15.065498 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec475cf-e0da-4f53-976c-44234bc65f7b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:15.075921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:15.105707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec475cf-e0da-4f53-976c-44234bc65f7b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:15.130847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b6cb8-41e0-4e66-a424-8965be2c9454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:15.131573 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b6cb8-41e0-4e66-a424-8965be2c9454 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:15.131766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b6cb8-41e0-4e66-a424-8965be2c9454 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:15.132050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b6cb8-41e0-4e66-a424-8965be2c9454 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:15.132740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b6cb8-41e0-4e66-a424-8965be2c9454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:15.171295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b6cb8-41e0-4e66-a424-8965be2c9454 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:15.230340 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 739/1464] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:15.237092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b6cb8-41e0-4e66-a424-8965be2c9454 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:15.244857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3357a4-0cfa-4b14-96b7-485bf391c656 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:15.270742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964046c-331b-4498-a67d-0ca47c145db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:15.273140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964046c-331b-4498-a67d-0ca47c145db6 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:15.273469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964046c-331b-4498-a67d-0ca47c145db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:15.273859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964046c-331b-4498-a67d-0ca47c145db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:15.274337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b6cb8-41e0-4e66-a424-8965be2c9454 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:15.274450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964046c-331b-4498-a67d-0ca47c145db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:15.283527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964046c-331b-4498-a67d-0ca47c145db6 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:15.291687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcf7e0-db78-4be4-ad19-e05f1945b2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:15.293117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcf7e0-db78-4be4-ad19-e05f1945b2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:15.293523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcf7e0-db78-4be4-ad19-e05f1945b2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:15.293684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcf7e0-db78-4be4-ad19-e05f1945b2ae admin admin] PATH_INFO: `/v3/projects/27802424c504421e85bfefd508bd85dc/users/43293b331c874be09b4314546a01d463/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:15.337689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcf7e0-db78-4be4-ad19-e05f1945b2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=27802424c504421e85bfefd508bd85dc, user_id=43293b331c874be09b4314546a01d463, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:15.344370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcf7e0-db78-4be4-ad19-e05f1945b2ae admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:15.364976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 726/1465] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:14 2022] POST /identity/v3/projects => generated 387 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:15.379019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964046c-331b-4498-a67d-0ca47c145db6 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:15.386207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 740/1466] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:15 2022] PUT /identity/v3/projects/27802424c504421e85bfefd508bd85dc/users/43293b331c874be09b4314546a01d463/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:15.391067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcf7e0-db78-4be4-ad19-e05f1945b2ae admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:15.591707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcf7e0-db78-4be4-ad19-e05f1945b2ae admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:15.604776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c029be9-7b60-4f05-859b-8416d2e5baee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:15.607002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c029be9-7b60-4f05-859b-8416d2e5baee admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:15.607002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c029be9-7b60-4f05-859b-8416d2e5baee admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:15.607002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c029be9-7b60-4f05-859b-8416d2e5baee admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:15.607479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c029be9-7b60-4f05-859b-8416d2e5baee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:15.613864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c029be9-7b60-4f05-859b-8416d2e5baee admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:15.616555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964046c-331b-4498-a67d-0ca47c145db6 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:15.633122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe74bb0-79fe-453b-933e-32cf5d66bb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:15.633518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe74bb0-79fe-453b-933e-32cf5d66bb61 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:15.633664 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe74bb0-79fe-453b-933e-32cf5d66bb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:15.633934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe74bb0-79fe-453b-933e-32cf5d66bb61 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:15.636872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe74bb0-79fe-453b-933e-32cf5d66bb61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:15.640460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe74bb0-79fe-453b-933e-32cf5d66bb61 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:15.641924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 741/1467] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:15.657958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c029be9-7b60-4f05-859b-8416d2e5baee admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:15.674998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c029be9-7b60-4f05-859b-8416d2e5baee admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:15.689326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7ec8f-4e24-4680-82e9-ef5adaeb4173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:15.689913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7ec8f-4e24-4680-82e9-ef5adaeb4173 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:15.690273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7ec8f-4e24-4680-82e9-ef5adaeb4173 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:15.690468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7ec8f-4e24-4680-82e9-ef5adaeb4173 admin admin] PATH_INFO: `/v3/projects/27802424c504421e85bfefd508bd85dc/users/43293b331c874be09b4314546a01d463/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:15.691804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 727/1468] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:15 2022] POST /identity/v3/users => generated 300 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:15.700674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe74bb0-79fe-453b-933e-32cf5d66bb61 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:15.702371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7ec8f-4e24-4680-82e9-ef5adaeb4173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27802424c504421e85bfefd508bd85dc, user_id=43293b331c874be09b4314546a01d463, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:15.716946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7ec8f-4e24-4680-82e9-ef5adaeb4173 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:15.723058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe74bb0-79fe-453b-933e-32cf5d66bb61 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:15.751658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59710ab5-3473-4dda-8394-9501a111f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:15.752379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59710ab5-3473-4dda-8394-9501a111f10e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:15.752517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 742/1469] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:15 2022] PUT /identity/v3/projects/27802424c504421e85bfefd508bd85dc/users/43293b331c874be09b4314546a01d463/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:15.752710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59710ab5-3473-4dda-8394-9501a111f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:15.752967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59710ab5-3473-4dda-8394-9501a111f10e admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:15.753561 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59710ab5-3473-4dda-8394-9501a111f10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:15.759399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59710ab5-3473-4dda-8394-9501a111f10e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:15.766479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdacbd-1fb1-4953-b4c2-9da0a045c33c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:15.767238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdacbd-1fb1-4953-b4c2-9da0a045c33c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:15.767238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdacbd-1fb1-4953-b4c2-9da0a045c33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:15.789694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 728/1470] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:15.794429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59710ab5-3473-4dda-8394-9501a111f10e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:15.824499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59710ab5-3473-4dda-8394-9501a111f10e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:15.836022 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f3fdacbd-1fb1-4953-b4c2-9da0a045c33c None None] MFA Rules not processed for user `43293b331c874be09b4314546a01d463`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:15.841193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f3fdacbd-1fb1-4953-b4c2-9da0a045c33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:16.075420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a88d0-fc45-445f-9171-7784a8d59422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:16.076501 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a88d0-fc45-445f-9171-7784a8d59422 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:16.076906 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a88d0-fc45-445f-9171-7784a8d59422 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:16.077388 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a88d0-fc45-445f-9171-7784a8d59422 admin admin] PATH_INFO: `/v3/projects/5df33c1494514c2280214df142a87712/users/a29b31bdbb68498aa74e4c94365bd175/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:16.083518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 743/1471] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:16.103876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdacbd-1fb1-4953-b4c2-9da0a045c33c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:16.112376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a88d0-fc45-445f-9171-7784a8d59422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df33c1494514c2280214df142a87712, user_id=a29b31bdbb68498aa74e4c94365bd175, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:16.121534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a88d0-fc45-445f-9171-7784a8d59422 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:16.147571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:16.171380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 729/1472] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:15 2022] PUT /identity/v3/projects/5df33c1494514c2280214df142a87712/users/a29b31bdbb68498aa74e4c94365bd175/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:16.173358 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a88d0-fc45-445f-9171-7784a8d59422 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:16.227861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:16.291279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdacbd-1fb1-4953-b4c2-9da0a045c33c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:16.311681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a88d0-fc45-445f-9171-7784a8d59422 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:16.318628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155ae03-2eba-4c8b-bbc6-93f4d88ad74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:16.322884 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155ae03-2eba-4c8b-bbc6-93f4d88ad74b admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:16.323255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155ae03-2eba-4c8b-bbc6-93f4d88ad74b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:16.325743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155ae03-2eba-4c8b-bbc6-93f4d88ad74b admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:16.326550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155ae03-2eba-4c8b-bbc6-93f4d88ad74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:16.331167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155ae03-2eba-4c8b-bbc6-93f4d88ad74b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:16.383967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 730/1473] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:16.389081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155ae03-2eba-4c8b-bbc6-93f4d88ad74b admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:16.415041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155ae03-2eba-4c8b-bbc6-93f4d88ad74b admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:16.435013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dd62d-a09d-4c2f-8f86-fc135c6779e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:16.435980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dd62d-a09d-4c2f-8f86-fc135c6779e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:16.436475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dd62d-a09d-4c2f-8f86-fc135c6779e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:16.436839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dd62d-a09d-4c2f-8f86-fc135c6779e3 admin admin] PATH_INFO: `/v3/projects/5df33c1494514c2280214df142a87712/users/a29b31bdbb68498aa74e4c94365bd175/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:16.439788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98dd62d-a09d-4c2f-8f86-fc135c6779e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df33c1494514c2280214df142a87712, user_id=a29b31bdbb68498aa74e4c94365bd175, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:16.445851 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98dd62d-a09d-4c2f-8f86-fc135c6779e3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:16.496923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958e449-6d53-45a8-967a-5b9dfd1a989a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:16.497561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958e449-6d53-45a8-967a-5b9dfd1a989a service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:16.497828 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958e449-6d53-45a8-967a-5b9dfd1a989a service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:16.498098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958e449-6d53-45a8-967a-5b9dfd1a989a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:16.499969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 731/1474] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:16 2022] PUT /identity/v3/projects/5df33c1494514c2280214df142a87712/users/a29b31bdbb68498aa74e4c94365bd175/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:16.502459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f9dfb-20e4-4469-80c1-6be1e5d70344 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:16.513918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f9dfb-20e4-4469-80c1-6be1e5d70344 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:16.515355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f9dfb-20e4-4469-80c1-6be1e5d70344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:16.558197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-684f9dfb-20e4-4469-80c1-6be1e5d70344 None None] MFA Rules not processed for user `4d74d7fd62cd464183f9df4ca581833c`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:16.559784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-684f9dfb-20e4-4469-80c1-6be1e5d70344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:16.682439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 732/1475] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:16.684638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f3bf3-299b-4226-a3dd-70df19e71beb None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:16.685334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958e449-6d53-45a8-967a-5b9dfd1a989a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:16.686130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f3bf3-299b-4226-a3dd-70df19e71beb None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:16.686612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f3bf3-299b-4226-a3dd-70df19e71beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:16.686802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958e449-6d53-45a8-967a-5b9dfd1a989a service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:16.715394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 744/1476] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:16.734193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-080f3bf3-299b-4226-a3dd-70df19e71beb None None] MFA Rules not processed for user `a29b31bdbb68498aa74e4c94365bd175`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:16.736089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-080f3bf3-299b-4226-a3dd-70df19e71beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:16.947262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 733/1477] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:17.010299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958e449-6d53-45a8-967a-5b9dfd1a989a service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:17.012397 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0958e449-6d53-45a8-967a-5b9dfd1a989a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:17.037252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958e449-6d53-45a8-967a-5b9dfd1a989a service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:17.280376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fe42f-c8fd-42ff-920e-b4a16c7c33cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:17.282703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fe42f-c8fd-42ff-920e-b4a16c7c33cc service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:17.286086 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fe42f-c8fd-42ff-920e-b4a16c7c33cc service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:17.286539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fe42f-c8fd-42ff-920e-b4a16c7c33cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:17.327009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fe42f-c8fd-42ff-920e-b4a16c7c33cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:17.331673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fe42f-c8fd-42ff-920e-b4a16c7c33cc service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:17.380999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 745/1478] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:19.159061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2dc74-c874-45ad-8ebe-26c251058106 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:19.160624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2dc74-c874-45ad-8ebe-26c251058106 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:19.161184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2dc74-c874-45ad-8ebe-26c251058106 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:19.163260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 734/1479] 10.222.0.13 () {54 vars in 912 bytes} [Thu Oct 20 14:41:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:41:19.182482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0285919-6cca-4a9e-a9f3-517b69a65845 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:19.183099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0285919-6cca-4a9e-a9f3-517b69a65845 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:19.183401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0285919-6cca-4a9e-a9f3-517b69a65845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:19.252560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c0285919-6cca-4a9e-a9f3-517b69a65845 None None] MFA Rules not processed for user `ae4b023dbf114bf98d0e38ea7fec94fa`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:19.256228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c0285919-6cca-4a9e-a9f3-517b69a65845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:19.305253 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:19.418427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2dc74-c874-45ad-8ebe-26c251058106 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:19.428326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 746/1480] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:41:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:19.476639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0285919-6cca-4a9e-a9f3-517b69a65845 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:19.489511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0285919-6cca-4a9e-a9f3-517b69a65845 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:19.497559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16ae4-2a69-4830-a42d-d8ecc03a2117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:19.498368 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16ae4-2a69-4830-a42d-d8ecc03a2117 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:19.506548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16ae4-2a69-4830-a42d-d8ecc03a2117 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:19.508157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16ae4-2a69-4830-a42d-d8ecc03a2117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:19.515161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:19.529577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16ae4-2a69-4830-a42d-d8ecc03a2117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:19.532242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16ae4-2a69-4830-a42d-d8ecc03a2117 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:19.542548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 747/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:19.602058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2dc74-c874-45ad-8ebe-26c251058106 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:19.607908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:19.609362 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:19.609632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:19.610249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] PATH_INFO: `/v3/users/0a99208281174f94b274c11caeb376e2` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:19.612133 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a99208281174f94b274c11caeb376e2)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:19.613837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:19.702519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a99208281174f94b274c11caeb376e2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:19.732335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a99208281174f94b274c11caeb376e2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:19.735420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a99208281174f94b274c11caeb376e2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:19.738651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a99208281174f94b274c11caeb376e2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:19.760192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a99208281174f94b274c11caeb376e2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:19.810220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 735/1482] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:19 2022] DELETE /identity/v3/users/0a99208281174f94b274c11caeb376e2 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:19.814478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e369f-8146-4006-8a4a-74611da3924a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:19.814748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e369f-8146-4006-8a4a-74611da3924a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:19.815025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e369f-8146-4006-8a4a-74611da3924a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:19.852832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1a0e369f-8146-4006-8a4a-74611da3924a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:19.856262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1a0e369f-8146-4006-8a4a-74611da3924a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:20.057092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 748/1483] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:20.095954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:20.097436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:20.256808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0888e-86c9-48b4-81c4-ef4972264e00 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:20.285191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f84b09-fd70-4039-8ba6-0c0963ca48fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:20.286075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f84b09-fd70-4039-8ba6-0c0963ca48fa service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:20.286502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f84b09-fd70-4039-8ba6-0c0963ca48fa service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:20.286910 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f84b09-fd70-4039-8ba6-0c0963ca48fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:20.314673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f84b09-fd70-4039-8ba6-0c0963ca48fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:20.317058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f84b09-fd70-4039-8ba6-0c0963ca48fa service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:20.331854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 736/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:20.451373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1905a9-cd47-45e4-beaf-e047e7071ef4 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:20.452324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1905a9-cd47-45e4-beaf-e047e7071ef4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:20.452665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1905a9-cd47-45e4-beaf-e047e7071ef4 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:20.454197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 749/1485] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:41:20.460554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461b22-dcca-4d23-8dc3-7626ba0b37ac None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:20.460870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461b22-dcca-4d23-8dc3-7626ba0b37ac None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:20.461208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461b22-dcca-4d23-8dc3-7626ba0b37ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:20.506476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-61461b22-dcca-4d23-8dc3-7626ba0b37ac None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:20.507811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-61461b22-dcca-4d23-8dc3-7626ba0b37ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:20.674240 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:20.742509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 737/1486] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:41:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:20.972656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1905a9-cd47-45e4-beaf-e047e7071ef4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:20.974824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1a1905a9-cd47-45e4-beaf-e047e7071ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:21.024574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1905a9-cd47-45e4-beaf-e047e7071ef4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:21.046734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:21.047910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:21.048874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:21.049655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] PATH_INFO: `/v3/users/fd3629e7e9ae43d38f4f1d1bbd76024b` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:21.059201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3629e7e9ae43d38f4f1d1bbd76024b)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:21.059396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461b22-dcca-4d23-8dc3-7626ba0b37ac None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:21.061294 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-61461b22-dcca-4d23-8dc3-7626ba0b37ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:21.062562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:21.083696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461b22-dcca-4d23-8dc3-7626ba0b37ac None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:21.099230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9003081-ed4c-428b-8fd5-2b0a529c3407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:21.103617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9003081-ed4c-428b-8fd5-2b0a529c3407 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:21.104008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9003081-ed4c-428b-8fd5-2b0a529c3407 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:21.104285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9003081-ed4c-428b-8fd5-2b0a529c3407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:21.137356 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9003081-ed4c-428b-8fd5-2b0a529c3407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:21.139934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9003081-ed4c-428b-8fd5-2b0a529c3407 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:21.156734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3629e7e9ae43d38f4f1d1bbd76024b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.173345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3629e7e9ae43d38f4f1d1bbd76024b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.175662 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 738/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:21.184416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3629e7e9ae43d38f4f1d1bbd76024b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.187328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3629e7e9ae43d38f4f1d1bbd76024b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.215736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3629e7e9ae43d38f4f1d1bbd76024b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.273464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 750/1488] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:20 2022] DELETE /identity/v3/users/fd3629e7e9ae43d38f4f1d1bbd76024b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:21.275326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6ff2c-652c-4d9c-93aa-2c1ab737a2b4 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:21.275862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6ff2c-652c-4d9c-93aa-2c1ab737a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:21.275862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6ff2c-652c-4d9c-93aa-2c1ab737a2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:21.279914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:21.281834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:21.313794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5ef6ff2c-652c-4d9c-93aa-2c1ab737a2b4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:21.314790 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5ef6ff2c-652c-4d9c-93aa-2c1ab737a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:21.366587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:21.454774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf69d05-d5bb-4a9d-ac41-6f109b85a466 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:21.462249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:21.463213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:21.463821 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:21.464242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] PATH_INFO: `/v3/projects/27fa1a3cd3504709a34b76f876d21b43` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:21.465614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] RBAC: Authorizing `identity:delete_project(project_id=27fa1a3cd3504709a34b76f876d21b43)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:21.468218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:21.503724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 739/1489] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:21.514948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6ff2c-652c-4d9c-93aa-2c1ab737a2b4 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:21.516874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5ef6ff2c-652c-4d9c-93aa-2c1ab737a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:21.543159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27fa1a3cd3504709a34b76f876d21b43', 'request_id': 'req-56d8e5de-826d-434f-93a4-227a40ef681a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.554414 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27fa1a3cd3504709a34b76f876d21b43', 'request_id': 'req-56d8e5de-826d-434f-93a4-227a40ef681a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.570146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27fa1a3cd3504709a34b76f876d21b43', 'request_id': 'req-56d8e5de-826d-434f-93a4-227a40ef681a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.588485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6ff2c-652c-4d9c-93aa-2c1ab737a2b4 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:21.603287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] The token cache is being invalidate because project 27fa1a3cd3504709a34b76f876d21b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:21.603533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.606367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.608425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 751/1490] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:21 2022] DELETE /identity/v3/projects/27fa1a3cd3504709a34b76f876d21b43 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:21.613432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:21.616061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:21.732218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8e5de-826d-434f-93a4-227a40ef681a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:21.746760 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:21.747432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:21.747714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:21.747995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] PATH_INFO: `/v3/users/a29b31bdbb68498aa74e4c94365bd175` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:21.757203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29b31bdbb68498aa74e4c94365bd175)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:21.759390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:21.805608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccd9e6-9127-42e8-aec6-61fa49263f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:21.805608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccd9e6-9127-42e8-aec6-61fa49263f12 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:21.806031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccd9e6-9127-42e8-aec6-61fa49263f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:21.806400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccd9e6-9127-42e8-aec6-61fa49263f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:21.809998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-52ccd9e6-9127-42e8-aec6-61fa49263f12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:21.829951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccd9e6-9127-42e8-aec6-61fa49263f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:21.836283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccd9e6-9127-42e8-aec6-61fa49263f12 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:21.861863 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29b31bdbb68498aa74e4c94365bd175'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.864596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 740/1491] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:21.882250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29b31bdbb68498aa74e4c94365bd175'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.883021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29b31bdbb68498aa74e4c94365bd175'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.886351 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29b31bdbb68498aa74e4c94365bd175'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.896853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29b31bdbb68498aa74e4c94365bd175'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:21.898866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:21.937861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 752/1492] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:21 2022] DELETE /identity/v3/users/a29b31bdbb68498aa74e4c94365bd175 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:22.628280 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccd9e6-9127-42e8-aec6-61fa49263f12 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:22.641190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccd9e6-9127-42e8-aec6-61fa49263f12 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:22.691140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:22.694801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:22.704587 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:22.745644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:22.746301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:22.746575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:22.746866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] PATH_INFO: `/v3/projects/5df33c1494514c2280214df142a87712` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:22.748128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df33c1494514c2280214df142a87712)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:22.750218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:22.773157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512f06c-2383-4697-8e96-0530273e56aa admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:22.778250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:22.778901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:22.779182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:22.779456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] PATH_INFO: `/v3/projects/5123134e4b3d4fa2a28b1fa77c7a403c` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:22.780607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5123134e4b3d4fa2a28b1fa77c7a403c)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:22.782526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:22.812699 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df33c1494514c2280214df142a87712', 'request_id': 'req-c5bb89de-d696-41cf-8036-e19a35f15cce'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.819922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df33c1494514c2280214df142a87712', 'request_id': 'req-c5bb89de-d696-41cf-8036-e19a35f15cce'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.830723 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df33c1494514c2280214df142a87712', 'request_id': 'req-c5bb89de-d696-41cf-8036-e19a35f15cce'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.843311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5123134e4b3d4fa2a28b1fa77c7a403c', 'request_id': 'req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.852481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5123134e4b3d4fa2a28b1fa77c7a403c', 'request_id': 'req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.855899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] The token cache is being invalidate because project 5df33c1494514c2280214df142a87712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:22.856270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.857023 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c5bb89de-d696-41cf-8036-e19a35f15cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.859122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 741/1493] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:22 2022] DELETE /identity/v3/projects/5df33c1494514c2280214df142a87712 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:22.864544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5123134e4b3d4fa2a28b1fa77c7a403c', 'request_id': 'req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.875552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a06c8-ee03-4c46-8404-1fb6c6d1ecea None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:22.875838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a06c8-ee03-4c46-8404-1fb6c6d1ecea None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:22.876112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a06c8-ee03-4c46-8404-1fb6c6d1ecea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:22.879499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] The token cache is being invalidate because project 5123134e4b3d4fa2a28b1fa77c7a403c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:22.880058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.880901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:22.886200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 753/1494] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:22 2022] DELETE /identity/v3/projects/5123134e4b3d4fa2a28b1fa77c7a403c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:22.890275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:22.891742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:22.900874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-2d3a06c8-ee03-4c46-8404-1fb6c6d1ecea None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:22.903155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2d3a06c8-ee03-4c46-8404-1fb6c6d1ecea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:22.968237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd978e-f9e8-4c76-bb4b-8e300abee08e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:22.975035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:22.975780 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:22.976139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:22.976518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] PATH_INFO: `/v3/users/43293b331c874be09b4314546a01d463` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:22.979685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=43293b331c874be09b4314546a01d463)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:22.980447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:22.997386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:23.000212 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 742/1495] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:23.012622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253aa0-fb1c-4e57-9863-c17b16dd411d None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:23.012935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253aa0-fb1c-4e57-9863-c17b16dd411d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:23.013273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253aa0-fb1c-4e57-9863-c17b16dd411d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:23.037126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-04253aa0-fb1c-4e57-9863-c17b16dd411d None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:23.038387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-04253aa0-fb1c-4e57-9863-c17b16dd411d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:23.053710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 743/1496] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:23.057691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253aa0-fb1c-4e57-9863-c17b16dd411d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:23.077655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253aa0-fb1c-4e57-9863-c17b16dd411d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:23.083535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7c86f-c7c4-4de9-9b47-351269b9e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:23.087694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7c86f-c7c4-4de9-9b47-351269b9e22d admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:23.088227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7c86f-c7c4-4de9-9b47-351269b9e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:23.089050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7c86f-c7c4-4de9-9b47-351269b9e22d admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:23.090056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7c86f-c7c4-4de9-9b47-351269b9e22d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:23.090414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7c86f-c7c4-4de9-9b47-351269b9e22d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:23.092045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7c86f-c7c4-4de9-9b47-351269b9e22d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:23.098437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43293b331c874be09b4314546a01d463'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:23.108524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 744/1497] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:23.113400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7c86f-c7c4-4de9-9b47-351269b9e22d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:23.117232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43293b331c874be09b4314546a01d463'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:23.118334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43293b331c874be09b4314546a01d463'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:23.124532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43293b331c874be09b4314546a01d463'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:23.125306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7c86f-c7c4-4de9-9b47-351269b9e22d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:23.130769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66946a-ae69-4302-8541-07e25bdb3a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:23.131497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66946a-ae69-4302-8541-07e25bdb3a66 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:23.131835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66946a-ae69-4302-8541-07e25bdb3a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:23.132176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66946a-ae69-4302-8541-07e25bdb3a66 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:23.132929 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66946a-ae69-4302-8541-07e25bdb3a66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:23.134984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66946a-ae69-4302-8541-07e25bdb3a66 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:23.138780 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43293b331c874be09b4314546a01d463'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:23.164083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 754/1498] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:22 2022] DELETE /identity/v3/users/43293b331c874be09b4314546a01d463 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:23.172426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 745/1499] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:23 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:23.174250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:23.175832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:23.316593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14895eca-309f-4065-9294-202433ce2a2e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:23.453810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74161a-17b7-4a6a-b590-2fcfac9a2861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:23.454535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74161a-17b7-4a6a-b590-2fcfac9a2861 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:23.454841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74161a-17b7-4a6a-b590-2fcfac9a2861 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:23.456266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74161a-17b7-4a6a-b590-2fcfac9a2861 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:23.457008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74161a-17b7-4a6a-b590-2fcfac9a2861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:23.462756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74161a-17b7-4a6a-b590-2fcfac9a2861 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:23.528323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 755/1500] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:23 2022] POST /identity/v3/users => generated 285 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:23.531541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66946a-ae69-4302-8541-07e25bdb3a66 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:23.542386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66946a-ae69-4302-8541-07e25bdb3a66 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:23.560679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac6169-d1c7-442f-97d5-3fb923133e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:23.561531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac6169-d1c7-442f-97d5-3fb923133e56 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:23.561971 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac6169-d1c7-442f-97d5-3fb923133e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:23.562452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac6169-d1c7-442f-97d5-3fb923133e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:23.563276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac6169-d1c7-442f-97d5-3fb923133e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:23.575402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac6169-d1c7-442f-97d5-3fb923133e56 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:23.606698 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 746/1501] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:23.610834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74161a-17b7-4a6a-b590-2fcfac9a2861 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:23.622391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74161a-17b7-4a6a-b590-2fcfac9a2861 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:23.628389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97857737-0c02-4a3b-b397-2ce0035bff9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:23.629049 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97857737-0c02-4a3b-b397-2ce0035bff9a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:23.629311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97857737-0c02-4a3b-b397-2ce0035bff9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:23.629590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97857737-0c02-4a3b-b397-2ce0035bff9a admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:23.630638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97857737-0c02-4a3b-b397-2ce0035bff9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:23.632062 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97857737-0c02-4a3b-b397-2ce0035bff9a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:23.646382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 756/1502] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:23.654807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac6169-d1c7-442f-97d5-3fb923133e56 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:23.686808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac6169-d1c7-442f-97d5-3fb923133e56 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:23.702063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dd6c5-e636-4d97-bd89-773d898c6d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:23.702756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dd6c5-e636-4d97-bd89-773d898c6d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:23.703038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dd6c5-e636-4d97-bd89-773d898c6d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:23.703315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dd6c5-e636-4d97-bd89-773d898c6d42 admin admin] PATH_INFO: `/v3/projects/d23b6c045d6843beaaeee602786f2737/users/900f27b4d0374ad79a2e3250a0c0a012/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:23.710121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:23.742041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dd6c5-e636-4d97-bd89-773d898c6d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23b6c045d6843beaaeee602786f2737, user_id=900f27b4d0374ad79a2e3250a0c0a012, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:23.757751 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dd6c5-e636-4d97-bd89-773d898c6d42 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:23.803238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 747/1503] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:23 2022] PUT /identity/v3/projects/d23b6c045d6843beaaeee602786f2737/users/900f27b4d0374ad79a2e3250a0c0a012/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:23.805297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b9dd3-5a5e-4217-9880-748096eccbaf None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:23.805798 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b9dd3-5a5e-4217-9880-748096eccbaf None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:23.805962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b9dd3-5a5e-4217-9880-748096eccbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:23.819331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dd6c5-e636-4d97-bd89-773d898c6d42 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:23.823760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-336dd6c5-e636-4d97-bd89-773d898c6d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:23.926722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-cb3b9dd3-5a5e-4217-9880-748096eccbaf None None] MFA Rules not processed for user `900f27b4d0374ad79a2e3250a0c0a012`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:23.927960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-cb3b9dd3-5a5e-4217-9880-748096eccbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:23.940750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:24.106344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dd6c5-e636-4d97-bd89-773d898c6d42 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:24.122433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:24.123233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:24.123709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:24.124194 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] PATH_INFO: `/v3/projects/27802424c504421e85bfefd508bd85dc` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:24.126688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] RBAC: Authorizing `identity:delete_project(project_id=27802424c504421e85bfefd508bd85dc)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:24.132225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:24.161883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 757/1504] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:24.181703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b9dd3-5a5e-4217-9880-748096eccbaf None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:24.184944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-cb3b9dd3-5a5e-4217-9880-748096eccbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:24.260222 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27802424c504421e85bfefd508bd85dc', 'request_id': 'req-7b7075be-e328-47b1-86de-7c467ce4048a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:24.282897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27802424c504421e85bfefd508bd85dc', 'request_id': 'req-7b7075be-e328-47b1-86de-7c467ce4048a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:24.315411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27802424c504421e85bfefd508bd85dc', 'request_id': 'req-7b7075be-e328-47b1-86de-7c467ce4048a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:24.365426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] The token cache is being invalidate because project 27802424c504421e85bfefd508bd85dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:24.365742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:24.368461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7b7075be-e328-47b1-86de-7c467ce4048a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:24.373925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 748/1505] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:23 2022] DELETE /identity/v3/projects/27802424c504421e85bfefd508bd85dc => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:24.386766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcd6fa-a11b-471f-ac7d-2c7c8ea8d46a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:24.387127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcd6fa-a11b-471f-ac7d-2c7c8ea8d46a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:24.387430 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcd6fa-a11b-471f-ac7d-2c7c8ea8d46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:24.460288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-c7bcd6fa-a11b-471f-ac7d-2c7c8ea8d46a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:24.463924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c7bcd6fa-a11b-471f-ac7d-2c7c8ea8d46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:24.503508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b9dd3-5a5e-4217-9880-748096eccbaf None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:24.627156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b0f61-b0a2-4663-a624-a28c93766cf3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cacfce5c9d07471cbe1713a4d6aa29f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:24.627625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b0f61-b0a2-4663-a624-a28c93766cf3 service glance] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:24.627909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b0f61-b0a2-4663-a624-a28c93766cf3 service glance] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:24.628185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b0f61-b0a2-4663-a624-a28c93766cf3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:24.633159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b07b0f61-b0a2-4663-a624-a28c93766cf3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:24.635806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 749/1506] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:24.646728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbc60f-4af3-4444-976b-31d90d8610e5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:24.646728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbc60f-4af3-4444-976b-31d90d8610e5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:24.646728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbc60f-4af3-4444-976b-31d90d8610e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:24.707184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-82cbc60f-4af3-4444-976b-31d90d8610e5 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:24.710895 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-82cbc60f-4af3-4444-976b-31d90d8610e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:24.730396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 750/1507] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:24.733092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbc60f-4af3-4444-976b-31d90d8610e5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:24.753126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbc60f-4af3-4444-976b-31d90d8610e5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:24.764675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f01065-8442-41a2-bc83-d9c0f843bf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:24.766368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f01065-8442-41a2-bc83-d9c0f843bf8c admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:24.767166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f01065-8442-41a2-bc83-d9c0f843bf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:24.768503 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f01065-8442-41a2-bc83-d9c0f843bf8c admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:24.769744 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f01065-8442-41a2-bc83-d9c0f843bf8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:24.770282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f01065-8442-41a2-bc83-d9c0f843bf8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:24.773874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f01065-8442-41a2-bc83-d9c0f843bf8c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:24.777748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:24.790381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:24.795246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b0f61-b0a2-4663-a624-a28c93766cf3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:24.798389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b0f61-b0a2-4663-a624-a28c93766cf3 service glance] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:24.799270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 751/1508] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:24.803450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f01065-8442-41a2-bc83-d9c0f843bf8c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:24.812105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 758/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:24.835401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f01065-8442-41a2-bc83-d9c0f843bf8c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:24.864803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33c833-22fe-4cc3-b942-06ddf06def84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:24.866179 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33c833-22fe-4cc3-b942-06ddf06def84 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:24.866669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33c833-22fe-4cc3-b942-06ddf06def84 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:24.867151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33c833-22fe-4cc3-b942-06ddf06def84 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:24.868052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33c833-22fe-4cc3-b942-06ddf06def84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:24.871002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33c833-22fe-4cc3-b942-06ddf06def84 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:24.911405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 752/1510] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:24 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:24.915711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b0f61-b0a2-4663-a624-a28c93766cf3 service glance] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:24.938202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b0f61-b0a2-4663-a624-a28c93766cf3 service glance] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:25.032302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e4f8c-96d0-43b2-835d-8bb2253714c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:25.033116 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e4f8c-96d0-43b2-835d-8bb2253714c5 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:25.033542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e4f8c-96d0-43b2-835d-8bb2253714c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:25.034021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e4f8c-96d0-43b2-835d-8bb2253714c5 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:25.034460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e4f8c-96d0-43b2-835d-8bb2253714c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:25.036879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e4f8c-96d0-43b2-835d-8bb2253714c5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:25.069394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 759/1511] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:24 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:25.075593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33c833-22fe-4cc3-b942-06ddf06def84 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:25.099285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33c833-22fe-4cc3-b942-06ddf06def84 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:25.122411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2516a4e-04b3-4aca-8cb8-5a42aa0bf374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:25.123142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2516a4e-04b3-4aca-8cb8-5a42aa0bf374 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:25.123492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2516a4e-04b3-4aca-8cb8-5a42aa0bf374 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:25.123837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2516a4e-04b3-4aca-8cb8-5a42aa0bf374 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:25.124580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2516a4e-04b3-4aca-8cb8-5a42aa0bf374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:25.126337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2516a4e-04b3-4aca-8cb8-5a42aa0bf374 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:25.176118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 753/1512] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:25.186682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e4f8c-96d0-43b2-835d-8bb2253714c5 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:25.197298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e4f8c-96d0-43b2-835d-8bb2253714c5 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:25.202787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301f1d0-18c6-4f16-8db3-1fce3a50071a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:25.203798 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301f1d0-18c6-4f16-8db3-1fce3a50071a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:25.204124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301f1d0-18c6-4f16-8db3-1fce3a50071a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:25.204463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301f1d0-18c6-4f16-8db3-1fce3a50071a admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:25.205159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301f1d0-18c6-4f16-8db3-1fce3a50071a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:25.206848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301f1d0-18c6-4f16-8db3-1fce3a50071a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:25.227886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 760/1513] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:25.230322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2516a4e-04b3-4aca-8cb8-5a42aa0bf374 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:25.248696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2516a4e-04b3-4aca-8cb8-5a42aa0bf374 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:25.254118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88742295-a628-4c0e-b17e-7286868babb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:25.255220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88742295-a628-4c0e-b17e-7286868babb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:25.255958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88742295-a628-4c0e-b17e-7286868babb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:25.256520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88742295-a628-4c0e-b17e-7286868babb5 admin admin] PATH_INFO: `/v3/projects/7df8b13bbfee41be9c8b84463bbc084c/users/b7993ef186d449b4ad933bdfa900635e/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:25.281297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88742295-a628-4c0e-b17e-7286868babb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df8b13bbfee41be9c8b84463bbc084c, user_id=b7993ef186d449b4ad933bdfa900635e, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:25.283959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88742295-a628-4c0e-b17e-7286868babb5 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:25.314552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 754/1514] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:25 2022] PUT /identity/v3/projects/7df8b13bbfee41be9c8b84463bbc084c/users/b7993ef186d449b4ad933bdfa900635e/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:25.330535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dc248-42d4-42d2-93a8-8437c0558a5e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:25.335358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dc248-42d4-42d2-93a8-8437c0558a5e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:25.335358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dc248-42d4-42d2-93a8-8437c0558a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:25.409773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-188dc248-42d4-42d2-93a8-8437c0558a5e None None] MFA Rules not processed for user `b7993ef186d449b4ad933bdfa900635e`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:25.411035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-188dc248-42d4-42d2-93a8-8437c0558a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:25.691453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88742295-a628-4c0e-b17e-7286868babb5 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:25.694361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 761/1515] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:25.934064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88742295-a628-4c0e-b17e-7286868babb5 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:25.957892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610eedd-12c0-420f-a095-d0ad0cae05e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:25.958840 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610eedd-12c0-420f-a095-d0ad0cae05e9 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:25.959593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610eedd-12c0-420f-a095-d0ad0cae05e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:25.960053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610eedd-12c0-420f-a095-d0ad0cae05e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:25.960904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610eedd-12c0-420f-a095-d0ad0cae05e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:25.970172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610eedd-12c0-420f-a095-d0ad0cae05e9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:26.051124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 755/1516] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:25 2022] POST /identity/v3/projects => generated 387 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:26.052409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dc248-42d4-42d2-93a8-8437c0558a5e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:26.075472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dc248-42d4-42d2-93a8-8437c0558a5e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:26.270054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031d20e-5c33-4de8-81af-8a0eb7241ef7 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:26.272490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031d20e-5c33-4de8-81af-8a0eb7241ef7 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:26.272490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031d20e-5c33-4de8-81af-8a0eb7241ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:26.310539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9db070-ed20-406d-9e50-795f369a845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:26.311475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9db070-ed20-406d-9e50-795f369a845b admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:26.312219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9db070-ed20-406d-9e50-795f369a845b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:26.312680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9db070-ed20-406d-9e50-795f369a845b admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:26.313628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9db070-ed20-406d-9e50-795f369a845b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:26.317437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9db070-ed20-406d-9e50-795f369a845b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:26.340148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-e031d20e-5c33-4de8-81af-8a0eb7241ef7 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:26.358258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e031d20e-5c33-4de8-81af-8a0eb7241ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:26.360783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:26.402639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 762/1517] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:26 2022] POST /identity/v3/users => generated 294 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:26.403515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9db070-ed20-406d-9e50-795f369a845b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:26.430766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9db070-ed20-406d-9e50-795f369a845b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:26.454262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8fa3f-dbb9-4c91-9c59-aae3e55971ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:26.454942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8fa3f-dbb9-4c91-9c59-aae3e55971ee admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:26.455293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8fa3f-dbb9-4c91-9c59-aae3e55971ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:26.455635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8fa3f-dbb9-4c91-9c59-aae3e55971ee admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:26.456366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8fa3f-dbb9-4c91-9c59-aae3e55971ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:26.463608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8fa3f-dbb9-4c91-9c59-aae3e55971ee admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:26.489933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 763/1518] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:26.489933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8fa3f-dbb9-4c91-9c59-aae3e55971ee admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:26.522604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8fa3f-dbb9-4c91-9c59-aae3e55971ee admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:26.545685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161c4fa-309d-4657-82bd-27bf9df2ad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:26.546641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161c4fa-309d-4657-82bd-27bf9df2ad85 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:26.546958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161c4fa-309d-4657-82bd-27bf9df2ad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:26.547272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161c4fa-309d-4657-82bd-27bf9df2ad85 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:26.547379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 756/1519] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:41:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:41:26.549373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161c4fa-309d-4657-82bd-27bf9df2ad85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:26.552145 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161c4fa-309d-4657-82bd-27bf9df2ad85 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:26.567303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031d20e-5c33-4de8-81af-8a0eb7241ef7 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:26.569102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e031d20e-5c33-4de8-81af-8a0eb7241ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:26.588003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 764/1520] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:26.594066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161c4fa-309d-4657-82bd-27bf9df2ad85 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:26.630288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:26.640731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161c4fa-309d-4657-82bd-27bf9df2ad85 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:26.656207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0e908-75b1-4f29-9d83-e86d6c722a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:26.658615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0e908-75b1-4f29-9d83-e86d6c722a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:26.659571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0e908-75b1-4f29-9d83-e86d6c722a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:26.659905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0e908-75b1-4f29-9d83-e86d6c722a5e admin admin] PATH_INFO: `/v3/projects/ef03e0cd57b8443790002e1e5e4e9aa9/users/6f775b0eb5a64d819f2195edd0c6865b/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:26.701112 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0e908-75b1-4f29-9d83-e86d6c722a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef03e0cd57b8443790002e1e5e4e9aa9, user_id=6f775b0eb5a64d819f2195edd0c6865b, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:26.704547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0e908-75b1-4f29-9d83-e86d6c722a5e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:26.720503 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031d20e-5c33-4de8-81af-8a0eb7241ef7 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:26.722420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 765/1521] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:26 2022] PUT /identity/v3/projects/ef03e0cd57b8443790002e1e5e4e9aa9/users/6f775b0eb5a64d819f2195edd0c6865b/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:26.736219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd80a-df3b-4488-8a9d-d3b64b40e98a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:26.736596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd80a-df3b-4488-8a9d-d3b64b40e98a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:26.736918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd80a-df3b-4488-8a9d-d3b64b40e98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:26.800940 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-889cd80a-df3b-4488-8a9d-d3b64b40e98a None None] MFA Rules not processed for user `6f775b0eb5a64d819f2195edd0c6865b`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:26.802147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-889cd80a-df3b-4488-8a9d-d3b64b40e98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:26.851683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c810704-379c-4af7-b54b-42919e1b74da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:26.852626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c810704-379c-4af7-b54b-42919e1b74da service swift] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:26.853105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c810704-379c-4af7-b54b-42919e1b74da service swift] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:26.853578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c810704-379c-4af7-b54b-42919e1b74da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:26.940702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c810704-379c-4af7-b54b-42919e1b74da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:26.942609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c810704-379c-4af7-b54b-42919e1b74da service swift] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:26.948939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 766/1522] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:26.949713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd80a-df3b-4488-8a9d-d3b64b40e98a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:26.953366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 757/1523] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:41:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:41:27.052741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd80a-df3b-4488-8a9d-d3b64b40e98a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:27.062343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab545f-062c-4899-af45-afe00a2143d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:27.067898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab545f-062c-4899-af45-afe00a2143d9 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:27.068422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab545f-062c-4899-af45-afe00a2143d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:27.068761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab545f-062c-4899-af45-afe00a2143d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:27.069472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab545f-062c-4899-af45-afe00a2143d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:27.074864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab545f-062c-4899-af45-afe00a2143d9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:27.154812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 767/1524] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:26 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:27.159198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c810704-379c-4af7-b54b-42919e1b74da service swift] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:27.177978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c810704-379c-4af7-b54b-42919e1b74da service swift] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:27.302117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950f46e-6993-4506-9e75-7c6c4e5a7978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:27.302870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950f46e-6993-4506-9e75-7c6c4e5a7978 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:27.303274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950f46e-6993-4506-9e75-7c6c4e5a7978 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:27.303619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950f46e-6993-4506-9e75-7c6c4e5a7978 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:27.304499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950f46e-6993-4506-9e75-7c6c4e5a7978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:27.306498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950f46e-6993-4506-9e75-7c6c4e5a7978 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:27.359863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:27.376299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 758/1525] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:27 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:27.378028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab545f-062c-4899-af45-afe00a2143d9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:27.392898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab545f-062c-4899-af45-afe00a2143d9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:27.400194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd44eed-4558-43a0-9588-47ddc28c0688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:27.401528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd44eed-4558-43a0-9588-47ddc28c0688 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:27.402683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd44eed-4558-43a0-9588-47ddc28c0688 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:27.403128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd44eed-4558-43a0-9588-47ddc28c0688 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:27.403871 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd44eed-4558-43a0-9588-47ddc28c0688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:27.408474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd44eed-4558-43a0-9588-47ddc28c0688 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:27.422301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 768/1526] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:27.423865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950f46e-6993-4506-9e75-7c6c4e5a7978 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:27.435124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950f46e-6993-4506-9e75-7c6c4e5a7978 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:27.440473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd566f65-1a9d-4663-9b22-d43e84d00982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:27.441164 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd566f65-1a9d-4663-9b22-d43e84d00982 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:27.441439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd566f65-1a9d-4663-9b22-d43e84d00982 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:27.441708 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd566f65-1a9d-4663-9b22-d43e84d00982 admin admin] PATH_INFO: `/v3/projects/ce1053f1a9e54f6aa9be2a49a1064e78/users/ef4f1a005c574736a9bc485470770c9a/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:27.465535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd566f65-1a9d-4663-9b22-d43e84d00982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1053f1a9e54f6aa9be2a49a1064e78, user_id=ef4f1a005c574736a9bc485470770c9a, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:27.469489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd566f65-1a9d-4663-9b22-d43e84d00982 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:27.487874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 759/1527] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:27 2022] PUT /identity/v3/projects/ce1053f1a9e54f6aa9be2a49a1064e78/users/ef4f1a005c574736a9bc485470770c9a/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:27.490769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd44eed-4558-43a0-9588-47ddc28c0688 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:27.504846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd44eed-4558-43a0-9588-47ddc28c0688 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:27.526096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:27.588608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd566f65-1a9d-4663-9b22-d43e84d00982 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:27.590917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cd566f65-1a9d-4663-9b22-d43e84d00982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:27.684534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d4c1-444a-4212-9bff-86075ba6c797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:27.685731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d4c1-444a-4212-9bff-86075ba6c797 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:27.686122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d4c1-444a-4212-9bff-86075ba6c797 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:27.686509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d4c1-444a-4212-9bff-86075ba6c797 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:27.687286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d4c1-444a-4212-9bff-86075ba6c797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:27.689677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d4c1-444a-4212-9bff-86075ba6c797 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:27.702101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 769/1528] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:27.707594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d4c1-444a-4212-9bff-86075ba6c797 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:27.722076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d4c1-444a-4212-9bff-86075ba6c797 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:27.727992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728738f2-6e92-4b32-96aa-465dd46c2914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:27.728766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728738f2-6e92-4b32-96aa-465dd46c2914 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:27.730829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728738f2-6e92-4b32-96aa-465dd46c2914 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:27.730829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728738f2-6e92-4b32-96aa-465dd46c2914 admin admin] PATH_INFO: `/v3/projects/ce1053f1a9e54f6aa9be2a49a1064e78/users/ef4f1a005c574736a9bc485470770c9a/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:27.732245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728738f2-6e92-4b32-96aa-465dd46c2914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1053f1a9e54f6aa9be2a49a1064e78, user_id=ef4f1a005c574736a9bc485470770c9a, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:27.735861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728738f2-6e92-4b32-96aa-465dd46c2914 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:27.751668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 770/1529] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:27 2022] PUT /identity/v3/projects/ce1053f1a9e54f6aa9be2a49a1064e78/users/ef4f1a005c574736a9bc485470770c9a/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:27.782875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2f501-5056-4b8e-88e3-ee2a88b14dd3 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:27.786592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2f501-5056-4b8e-88e3-ee2a88b14dd3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:27.786962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2f501-5056-4b8e-88e3-ee2a88b14dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:27.853807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-6bd2f501-5056-4b8e-88e3-ee2a88b14dd3 None None] MFA Rules not processed for user `ef4f1a005c574736a9bc485470770c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:27.856103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6bd2f501-5056-4b8e-88e3-ee2a88b14dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:27.867576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd566f65-1a9d-4663-9b22-d43e84d00982 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:28.010013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 771/1530] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:28.012543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2f501-5056-4b8e-88e3-ee2a88b14dd3 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:28.023684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6bd2f501-5056-4b8e-88e3-ee2a88b14dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:28.085840 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:28.086648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:28.086930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:28.087875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] PATH_INFO: `/v3/users/f12e4ec3623540cdb5018a10a9c52337` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:28.089751 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f12e4ec3623540cdb5018a10a9c52337)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:28.096047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:28.179109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2f501-5056-4b8e-88e3-ee2a88b14dd3 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:28.185579 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:28.186521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:28.187034 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:28.187434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] PATH_INFO: `/v3/users/b12517a82e67441ba87a79adc7c2ee93` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:28.188713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12517a82e67441ba87a79adc7c2ee93)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:28.191152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:28.207890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f12e4ec3623540cdb5018a10a9c52337'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.219652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:28.241143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f12e4ec3623540cdb5018a10a9c52337'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.241982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f12e4ec3623540cdb5018a10a9c52337'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.242623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f12e4ec3623540cdb5018a10a9c52337'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.265507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f12e4ec3623540cdb5018a10a9c52337'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.280654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12517a82e67441ba87a79adc7c2ee93'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.297082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12517a82e67441ba87a79adc7c2ee93'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.298337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12517a82e67441ba87a79adc7c2ee93'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.300671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12517a82e67441ba87a79adc7c2ee93'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.303991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 760/1531] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:27 2022] DELETE /identity/v3/users/f12e4ec3623540cdb5018a10a9c52337 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:28.305097 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:28.308364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:28.314377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:28.314568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-29be4c2b-7a3d-4b49-b81a-4b71b27a53ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12517a82e67441ba87a79adc7c2ee93'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:28.343757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 772/1532] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:28 2022] DELETE /identity/v3/users/b12517a82e67441ba87a79adc7c2ee93 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:28.345585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed4e1f-6fb2-4093-a17b-feaa70aa4ba0 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:28.346053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed4e1f-6fb2-4093-a17b-feaa70aa4ba0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:28.346485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed4e1f-6fb2-4093-a17b-feaa70aa4ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:28.367566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-28ed4e1f-6fb2-4093-a17b-feaa70aa4ba0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:28.369476 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-28ed4e1f-6fb2-4093-a17b-feaa70aa4ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:28.378352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735b080-009d-4387-b94d-8e0e5e7cff01 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:28.441188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3ef04-9d87-42cf-ad3d-43a85e9cf0bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:28.441654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3ef04-9d87-42cf-ad3d-43a85e9cf0bc service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:28.441940 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3ef04-9d87-42cf-ad3d-43a85e9cf0bc service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:28.442221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3ef04-9d87-42cf-ad3d-43a85e9cf0bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:28.444303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b0c3ef04-9d87-42cf-ad3d-43a85e9cf0bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:28.457872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 773/1533] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:28.457872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a171c-ae55-421a-9289-55f2ce54b251 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:28.457872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a171c-ae55-421a-9289-55f2ce54b251 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:28.457872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a171c-ae55-421a-9289-55f2ce54b251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:28.479660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-7c4a171c-ae55-421a-9289-55f2ce54b251 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:28.480942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7c4a171c-ae55-421a-9289-55f2ce54b251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:28.492179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 774/1534] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:28.492179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a171c-ae55-421a-9289-55f2ce54b251 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:28.492179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7c4a171c-ae55-421a-9289-55f2ce54b251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:28.542214 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3ef04-9d87-42cf-ad3d-43a85e9cf0bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:28.543835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3ef04-9d87-42cf-ad3d-43a85e9cf0bc service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:28.555532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 761/1535] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:28.556314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3ef04-9d87-42cf-ad3d-43a85e9cf0bc service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:28.567965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3ef04-9d87-42cf-ad3d-43a85e9cf0bc service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:28.573050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fbc9e-35ca-48c9-8e85-1c83e438d63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:28.573870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fbc9e-35ca-48c9-8e85-1c83e438d63f service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:28.574157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fbc9e-35ca-48c9-8e85-1c83e438d63f service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:28.574608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fbc9e-35ca-48c9-8e85-1c83e438d63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:28.578084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a171c-ae55-421a-9289-55f2ce54b251 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:28.583421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942978d2-2a91-42dd-9a4d-fc0bb612df1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:28.584093 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942978d2-2a91-42dd-9a4d-fc0bb612df1d service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:28.585018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942978d2-2a91-42dd-9a4d-fc0bb612df1d service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:28.585018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942978d2-2a91-42dd-9a4d-fc0bb612df1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:28.585434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fbc9e-35ca-48c9-8e85-1c83e438d63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:28.587619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fbc9e-35ca-48c9-8e85-1c83e438d63f service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:28.598699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942978d2-2a91-42dd-9a4d-fc0bb612df1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:28.600399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942978d2-2a91-42dd-9a4d-fc0bb612df1d service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:28.603469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 762/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:28.614050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 775/1537] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:29.516566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fbc9e-35ca-48c9-8e85-1c83e438d63f service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:29.518785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-783fbc9e-35ca-48c9-8e85-1c83e438d63f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:29.533073 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fbc9e-35ca-48c9-8e85-1c83e438d63f service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:29.538306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:29.539052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:29.539426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:29.539799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:29.542813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:29.640393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:29.650148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:29.653214 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:29.680087 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 763/1538] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:29.731278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942978d2-2a91-42dd-9a4d-fc0bb612df1d service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:29.735794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-942978d2-2a91-42dd-9a4d-fc0bb612df1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:29.766695 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:29.768260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:29.797638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc42b4f-e4ac-4769-96ac-77fbb10f8db0 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:29.811248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:29.814249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:29.815330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:29.816048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] PATH_INFO: `/v3/projects/0b80f77da11e430da8f615d4bbf74063` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:29.817453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942978d2-2a91-42dd-9a4d-fc0bb612df1d service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:29.818294 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b80f77da11e430da8f615d4bbf74063)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:29.823961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:29.840722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:29.841565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:29.842107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:29.842592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] PATH_INFO: `/v3/projects/ae7373c67edc493584fd03e50e54a8d0` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:29.844076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7373c67edc493584fd03e50e54a8d0)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:29.846273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:29.929238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:29.948208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b80f77da11e430da8f615d4bbf74063', 'request_id': 'req-24a479a3-916c-456b-a8c1-82bf415b6def'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:29.950471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7373c67edc493584fd03e50e54a8d0', 'request_id': 'req-f65868a0-e392-4871-ba71-199cc1e51c82'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:29.957154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7373c67edc493584fd03e50e54a8d0', 'request_id': 'req-f65868a0-e392-4871-ba71-199cc1e51c82'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:29.966632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b80f77da11e430da8f615d4bbf74063', 'request_id': 'req-24a479a3-916c-456b-a8c1-82bf415b6def'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:29.975712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7373c67edc493584fd03e50e54a8d0', 'request_id': 'req-f65868a0-e392-4871-ba71-199cc1e51c82'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:29.983198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b80f77da11e430da8f615d4bbf74063', 'request_id': 'req-24a479a3-916c-456b-a8c1-82bf415b6def'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.002097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] The token cache is being invalidate because project ae7373c67edc493584fd03e50e54a8d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:30.002311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.003100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.003509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] The token cache is being invalidate because project 0b80f77da11e430da8f615d4bbf74063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:30.003815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.004515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.005222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 776/1539] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:29 2022] DELETE /identity/v3/projects/ae7373c67edc493584fd03e50e54a8d0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:30.009255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:30.009431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 764/1540] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:29 2022] DELETE /identity/v3/projects/0b80f77da11e430da8f615d4bbf74063 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:30.014125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:30.014681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:30.015668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:30.193583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a479a3-916c-456b-a8c1-82bf415b6def admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:30.208718 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:30.209160 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:30.209445 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:30.209722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] PATH_INFO: `/v3/users/7f07a194efc14c9eb56eb25f7c8c965a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:30.210912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f07a194efc14c9eb56eb25f7c8c965a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:30.215742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:30.253466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65868a0-e392-4871-ba71-199cc1e51c82 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:30.279622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:30.280274 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:30.280819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:30.281097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] PATH_INFO: `/v3/users/a7377262284240d3ad4e3f34209ba360` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:30.283396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7377262284240d3ad4e3f34209ba360)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:30.285896 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:30.346747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f07a194efc14c9eb56eb25f7c8c965a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.368616 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f07a194efc14c9eb56eb25f7c8c965a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.374700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f07a194efc14c9eb56eb25f7c8c965a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.381191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f07a194efc14c9eb56eb25f7c8c965a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.394639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f1fdbbf6-94e5-4ee6-b1e2-ee7ee8c35465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f07a194efc14c9eb56eb25f7c8c965a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.415206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:30.433088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 765/1541] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:30 2022] DELETE /identity/v3/users/7f07a194efc14c9eb56eb25f7c8c965a => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:30.439272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7377262284240d3ad4e3f34209ba360'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.448209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7377262284240d3ad4e3f34209ba360'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.449294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7377262284240d3ad4e3f34209ba360'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.450003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7377262284240d3ad4e3f34209ba360'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.459852 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a58b802-1f39-453a-9456-4972e6c0add1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7377262284240d3ad4e3f34209ba360'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:30.485295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 777/1542] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:30 2022] DELETE /identity/v3/users/a7377262284240d3ad4e3f34209ba360 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:31.211131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac73c02-ec91-4481-b716-e73a01f1f890 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:31.211671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac73c02-ec91-4481-b716-e73a01f1f890 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:31.211671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac73c02-ec91-4481-b716-e73a01f1f890 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:31.213306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 766/1543] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:41:31.271774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e34447-d84e-4732-a87c-ca24183374e2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:31.272835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e34447-d84e-4732-a87c-ca24183374e2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:31.273186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e34447-d84e-4732-a87c-ca24183374e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:31.300063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-99e34447-d84e-4732-a87c-ca24183374e2 None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:31.301814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-99e34447-d84e-4732-a87c-ca24183374e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:31.377376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:31.530558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 778/1544] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:41:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:31.579101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac73c02-ec91-4481-b716-e73a01f1f890 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:31.582814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2ac73c02-ec91-4481-b716-e73a01f1f890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:31.616239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac73c02-ec91-4481-b716-e73a01f1f890 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:31.625343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a6350-acfa-4556-abcb-8840b92a30ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:31.626181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a6350-acfa-4556-abcb-8840b92a30ec service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:31.626516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a6350-acfa-4556-abcb-8840b92a30ec service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:31.626860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a6350-acfa-4556-abcb-8840b92a30ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:31.639198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a6350-acfa-4556-abcb-8840b92a30ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:31.640985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a6350-acfa-4556-abcb-8840b92a30ec service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:31.676374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 767/1545] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:31.902585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e34447-d84e-4732-a87c-ca24183374e2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:31.904172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-99e34447-d84e-4732-a87c-ca24183374e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:31.930360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a6350-acfa-4556-abcb-8840b92a30ec service nova] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:31.931830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-809a6350-acfa-4556-abcb-8840b92a30ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:32.038537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e34447-d84e-4732-a87c-ca24183374e2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.046273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.047272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.047847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.048121 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] PATH_INFO: `/v3/projects/f10f0aa4929d465cbe5dbe6807f6f080` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.049394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10f0aa4929d465cbe5dbe6807f6f080)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.051752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.077673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a6350-acfa-4556-abcb-8840b92a30ec service nova] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.092518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.095253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.095625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.096435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] PATH_INFO: `/v3/projects/b0914029e60343d8a4921a3e3cf52067` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.099601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0914029e60343d8a4921a3e3cf52067)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.106473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.154540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10f0aa4929d465cbe5dbe6807f6f080', 'request_id': 'req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.161132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10f0aa4929d465cbe5dbe6807f6f080', 'request_id': 'req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.175875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10f0aa4929d465cbe5dbe6807f6f080', 'request_id': 'req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.193862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] The token cache is being invalidate because project f10f0aa4929d465cbe5dbe6807f6f080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:32.194326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.196029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85dfec00-0d6f-4c35-a78f-37f7e5bb22ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.204245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 779/1546] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:31 2022] DELETE /identity/v3/projects/f10f0aa4929d465cbe5dbe6807f6f080 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:32.206193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a16166-d44e-4791-bb58-eac66707bbd1 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.206549 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a16166-d44e-4791-bb58-eac66707bbd1 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.206890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a16166-d44e-4791-bb58-eac66707bbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.252314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0914029e60343d8a4921a3e3cf52067', 'request_id': 'req-03a37183-07cb-4fd7-a278-b5f86b517a70'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.259783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0914029e60343d8a4921a3e3cf52067', 'request_id': 'req-03a37183-07cb-4fd7-a278-b5f86b517a70'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.270478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-71a16166-d44e-4791-bb58-eac66707bbd1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:32.272806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0914029e60343d8a4921a3e3cf52067', 'request_id': 'req-03a37183-07cb-4fd7-a278-b5f86b517a70'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.279698 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-71a16166-d44e-4791-bb58-eac66707bbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:32.305765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] The token cache is being invalidate because project b0914029e60343d8a4921a3e3cf52067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:32.308913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.311793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-03a37183-07cb-4fd7-a278-b5f86b517a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:32.315435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 768/1547] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:31 2022] DELETE /identity/v3/projects/b0914029e60343d8a4921a3e3cf52067 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:32.321150 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9e3cd-f410-4e5a-b6ac-cc140cb65916 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.321675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9e3cd-f410-4e5a-b6ac-cc140cb65916 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.322129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9e3cd-f410-4e5a-b6ac-cc140cb65916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.374903 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-f8d9e3cd-f410-4e5a-b6ac-cc140cb65916 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:32.378004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f8d9e3cd-f410-4e5a-b6ac-cc140cb65916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:32.401765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:32.426432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 780/1548] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:32.436146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88896d-b1b8-495a-a55b-35223b401510 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.436505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88896d-b1b8-495a-a55b-35223b401510 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.436943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88896d-b1b8-495a-a55b-35223b401510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.460251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-aa88896d-b1b8-495a-a55b-35223b401510 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:32.461538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-aa88896d-b1b8-495a-a55b-35223b401510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:32.471937 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 781/1549] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:41:32.475940 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88896d-b1b8-495a-a55b-35223b401510 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:32.476681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 769/1550] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:32.486586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83eee4-8945-4afb-a72d-d82cc28f0384 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.487144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83eee4-8945-4afb-a72d-d82cc28f0384 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.487611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83eee4-8945-4afb-a72d-d82cc28f0384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.516041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88896d-b1b8-495a-a55b-35223b401510 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.531994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eb649-12a5-4fe8-95eb-031574c11ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.533362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eb649-12a5-4fe8-95eb-031574c11ddf admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.534008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eb649-12a5-4fe8-95eb-031574c11ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.544027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-bc83eee4-8945-4afb-a72d-d82cc28f0384 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:32.547560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bc83eee4-8945-4afb-a72d-d82cc28f0384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:32.549483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eb649-12a5-4fe8-95eb-031574c11ddf admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.550215 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37eb649-12a5-4fe8-95eb-031574c11ddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:32.550537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37eb649-12a5-4fe8-95eb-031574c11ddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.552246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37eb649-12a5-4fe8-95eb-031574c11ddf admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.576756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 770/1551] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:32.583502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 782/1552] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:32.583608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83eee4-8945-4afb-a72d-d82cc28f0384 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:32.584486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eb649-12a5-4fe8-95eb-031574c11ddf admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:32.595629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83eee4-8945-4afb-a72d-d82cc28f0384 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.597746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eb649-12a5-4fe8-95eb-031574c11ddf admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.602156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b42c8-a60a-485c-8342-f67b1c9b2029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.602844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b42c8-a60a-485c-8342-f67b1c9b2029 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.603128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b42c8-a60a-485c-8342-f67b1c9b2029 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.605155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb9ef-7bf8-4a39-8d0e-91d4d6a47d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.605895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb9ef-7bf8-4a39-8d0e-91d4d6a47d63 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.606266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb9ef-7bf8-4a39-8d0e-91d4d6a47d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.606691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb9ef-7bf8-4a39-8d0e-91d4d6a47d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.607475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bb9ef-7bf8-4a39-8d0e-91d4d6a47d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.611349 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b42c8-a60a-485c-8342-f67b1c9b2029 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.612406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b42c8-a60a-485c-8342-f67b1c9b2029 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:32.612692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b42c8-a60a-485c-8342-f67b1c9b2029 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.614451 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b42c8-a60a-485c-8342-f67b1c9b2029 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.617049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:32.627317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bb9ef-7bf8-4a39-8d0e-91d4d6a47d63 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.636981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 771/1553] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:32.642835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b42c8-a60a-485c-8342-f67b1c9b2029 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:32.662742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b42c8-a60a-485c-8342-f67b1c9b2029 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.670129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abda6c6-1245-41c0-a24e-f83944899b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.670817 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 783/1554] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:32 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:32.670970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abda6c6-1245-41c0-a24e-f83944899b30 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.671375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abda6c6-1245-41c0-a24e-f83944899b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.671739 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abda6c6-1245-41c0-a24e-f83944899b30 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.672519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abda6c6-1245-41c0-a24e-f83944899b30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.674696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abda6c6-1245-41c0-a24e-f83944899b30 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.676464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb9ef-7bf8-4a39-8d0e-91d4d6a47d63 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:32.724114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 772/1555] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:32 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:32.728609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abda6c6-1245-41c0-a24e-f83944899b30 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:32.798774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb9ef-7bf8-4a39-8d0e-91d4d6a47d63 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.818693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abda6c6-1245-41c0-a24e-f83944899b30 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.824681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c268dbb-73f5-41df-9598-4dd3a61a2680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.825388 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c268dbb-73f5-41df-9598-4dd3a61a2680 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.825726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c268dbb-73f5-41df-9598-4dd3a61a2680 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.826013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c268dbb-73f5-41df-9598-4dd3a61a2680 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.826655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c268dbb-73f5-41df-9598-4dd3a61a2680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.839657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c268dbb-73f5-41df-9598-4dd3a61a2680 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.870394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077990c8-7c77-4c7f-bb23-b671f836c647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.871143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077990c8-7c77-4c7f-bb23-b671f836c647 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.871497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077990c8-7c77-4c7f-bb23-b671f836c647 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.871835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077990c8-7c77-4c7f-bb23-b671f836c647 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.873649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077990c8-7c77-4c7f-bb23-b671f836c647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.878032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 773/1556] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:32 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:32.879185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c268dbb-73f5-41df-9598-4dd3a61a2680 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:32.879942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077990c8-7c77-4c7f-bb23-b671f836c647 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.894290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c268dbb-73f5-41df-9598-4dd3a61a2680 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.902303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545e0f-d03d-48de-a822-7928c2b8e4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.903009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545e0f-d03d-48de-a822-7928c2b8e4ac admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.905095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545e0f-d03d-48de-a822-7928c2b8e4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.905414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545e0f-d03d-48de-a822-7928c2b8e4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.906176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28545e0f-d03d-48de-a822-7928c2b8e4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.908034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28545e0f-d03d-48de-a822-7928c2b8e4ac admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.913723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 784/1557] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:32 2022] POST /identity/v3/users => generated 288 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:32.918973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077990c8-7c77-4c7f-bb23-b671f836c647 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:32.931190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 774/1558] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:32.935405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545e0f-d03d-48de-a822-7928c2b8e4ac admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:32.941095 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077990c8-7c77-4c7f-bb23-b671f836c647 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.949364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545e0f-d03d-48de-a822-7928c2b8e4ac admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:32.954832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221de96a-e949-4763-b411-69c659423c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.956055 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221de96a-e949-4763-b411-69c659423c29 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.956524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221de96a-e949-4763-b411-69c659423c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.956933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221de96a-e949-4763-b411-69c659423c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.971830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221de96a-e949-4763-b411-69c659423c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.974907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221de96a-e949-4763-b411-69c659423c29 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.978614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6c323-9faf-4602-bfa7-1b5c5a2f0d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:32.979802 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6c323-9faf-4602-bfa7-1b5c5a2f0d75 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:32.980294 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6c323-9faf-4602-bfa7-1b5c5a2f0d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:32.981184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6c323-9faf-4602-bfa7-1b5c5a2f0d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:32.981299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6c323-9faf-4602-bfa7-1b5c5a2f0d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:32.983390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6c323-9faf-4602-bfa7-1b5c5a2f0d75 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:32.999401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 775/1559] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:33.004545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6c323-9faf-4602-bfa7-1b5c5a2f0d75 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.016868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6c323-9faf-4602-bfa7-1b5c5a2f0d75 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.023295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a91b6f-8763-47c2-ae8b-1c087d7e1103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.024119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91b6f-8763-47c2-ae8b-1c087d7e1103 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.024631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91b6f-8763-47c2-ae8b-1c087d7e1103 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.024789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91b6f-8763-47c2-ae8b-1c087d7e1103 admin admin] PATH_INFO: `/v3/projects/7f4f2c29f23846b7804baf6f95892907/users/31129b01036f49fdb1a2f20af970db9b/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.027179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 785/1560] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:33.029895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221de96a-e949-4763-b411-69c659423c29 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.048555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a91b6f-8763-47c2-ae8b-1c087d7e1103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4f2c29f23846b7804baf6f95892907, user_id=31129b01036f49fdb1a2f20af970db9b, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.052674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a91b6f-8763-47c2-ae8b-1c087d7e1103 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.059576 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221de96a-e949-4763-b411-69c659423c29 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.082263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544194f-7254-4a53-8044-346cc3c2f383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.082940 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544194f-7254-4a53-8044-346cc3c2f383 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.083222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544194f-7254-4a53-8044-346cc3c2f383 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.083498 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544194f-7254-4a53-8044-346cc3c2f383 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.084248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544194f-7254-4a53-8044-346cc3c2f383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.092715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544194f-7254-4a53-8044-346cc3c2f383 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.139718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 786/1561] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:33.142804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544194f-7254-4a53-8044-346cc3c2f383 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.151094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 776/1562] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:33 2022] PUT /identity/v3/projects/7f4f2c29f23846b7804baf6f95892907/users/31129b01036f49fdb1a2f20af970db9b/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:33.155092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4560e-e6df-42ee-ad46-12d676812d54 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.155577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4560e-e6df-42ee-ad46-12d676812d54 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.157175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4560e-e6df-42ee-ad46-12d676812d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.174519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544194f-7254-4a53-8044-346cc3c2f383 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.229452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-32f4560e-e6df-42ee-ad46-12d676812d54 None None] MFA Rules not processed for user `31129b01036f49fdb1a2f20af970db9b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:33.231070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-32f4560e-e6df-42ee-ad46-12d676812d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:33.231917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:33.279439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172752d-fc9a-4f9d-97e9-54ad5107da33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.280406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172752d-fc9a-4f9d-97e9-54ad5107da33 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.281026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172752d-fc9a-4f9d-97e9-54ad5107da33 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.281114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172752d-fc9a-4f9d-97e9-54ad5107da33 admin admin] PATH_INFO: `/v3/projects/7ab47d6fe38b45dbb97fe34b2dacac6b/users/06a10bcd35d34d1e922b30db9708f297/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.303869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172752d-fc9a-4f9d-97e9-54ad5107da33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab47d6fe38b45dbb97fe34b2dacac6b, user_id=06a10bcd35d34d1e922b30db9708f297, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.307763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172752d-fc9a-4f9d-97e9-54ad5107da33 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.326431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 787/1563] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:33 2022] PUT /identity/v3/projects/7ab47d6fe38b45dbb97fe34b2dacac6b/users/06a10bcd35d34d1e922b30db9708f297/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:33.330881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 777/1564] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:33.345034 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172752d-fc9a-4f9d-97e9-54ad5107da33 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.352439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a2677-104b-4f5a-b361-f99eb07aa636 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.354377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a2677-104b-4f5a-b361-f99eb07aa636 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.355452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a2677-104b-4f5a-b361-f99eb07aa636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.372577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:33.436039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-392a2677-104b-4f5a-b361-f99eb07aa636 None None] MFA Rules not processed for user `06a10bcd35d34d1e922b30db9708f297`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:33.437467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-392a2677-104b-4f5a-b361-f99eb07aa636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:33.454132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172752d-fc9a-4f9d-97e9-54ad5107da33 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.459221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd74a07-7a22-4a9c-8dcb-c67a830836ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.459838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd74a07-7a22-4a9c-8dcb-c67a830836ef admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.460130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd74a07-7a22-4a9c-8dcb-c67a830836ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.460538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd74a07-7a22-4a9c-8dcb-c67a830836ef admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.469213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd74a07-7a22-4a9c-8dcb-c67a830836ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.471274 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd74a07-7a22-4a9c-8dcb-c67a830836ef admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.527772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 788/1565] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:33 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:33.529378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd74a07-7a22-4a9c-8dcb-c67a830836ef admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.530396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 778/1566] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:33.536598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a2677-104b-4f5a-b361-f99eb07aa636 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.548867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a2677-104b-4f5a-b361-f99eb07aa636 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.631253 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd74a07-7a22-4a9c-8dcb-c67a830836ef admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.632981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c125e8-8499-45cc-bddd-00390ad75e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.633818 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c125e8-8499-45cc-bddd-00390ad75e41 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.634228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c125e8-8499-45cc-bddd-00390ad75e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.634743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c125e8-8499-45cc-bddd-00390ad75e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.636116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c125e8-8499-45cc-bddd-00390ad75e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.638559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c125e8-8499-45cc-bddd-00390ad75e41 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.644996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14019-7b5e-4b1a-9bcd-62b4888df184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.645627 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14019-7b5e-4b1a-9bcd-62b4888df184 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.645917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14019-7b5e-4b1a-9bcd-62b4888df184 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.646195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14019-7b5e-4b1a-9bcd-62b4888df184 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.647800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14019-7b5e-4b1a-9bcd-62b4888df184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.650204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14019-7b5e-4b1a-9bcd-62b4888df184 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.679087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 789/1567] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:33 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:33.682773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 779/1568] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:33 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:33.685431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14019-7b5e-4b1a-9bcd-62b4888df184 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.686480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c125e8-8499-45cc-bddd-00390ad75e41 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.695723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14019-7b5e-4b1a-9bcd-62b4888df184 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.729783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:33.762032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c125e8-8499-45cc-bddd-00390ad75e41 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.764634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2154e0-7735-4a0a-8519-b90a4c8c05cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.765433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2154e0-7735-4a0a-8519-b90a4c8c05cb admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.765775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2154e0-7735-4a0a-8519-b90a4c8c05cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.766183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2154e0-7735-4a0a-8519-b90a4c8c05cb admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.767420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b94958-5836-4469-b51b-b807aa0fa585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.767559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2154e0-7735-4a0a-8519-b90a4c8c05cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.767672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b94958-5836-4469-b51b-b807aa0fa585 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.767782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b94958-5836-4469-b51b-b807aa0fa585 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.768035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b94958-5836-4469-b51b-b807aa0fa585 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.768692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b94958-5836-4469-b51b-b807aa0fa585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.769079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2154e0-7735-4a0a-8519-b90a4c8c05cb admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.770752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b94958-5836-4469-b51b-b807aa0fa585 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.782336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 790/1569] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:33.785431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2154e0-7735-4a0a-8519-b90a4c8c05cb admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.801868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2154e0-7735-4a0a-8519-b90a4c8c05cb admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.808890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf418d7-b022-44dd-b2c5-6ee2039ee54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.809748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf418d7-b022-44dd-b2c5-6ee2039ee54b admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.810096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf418d7-b022-44dd-b2c5-6ee2039ee54b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.810437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf418d7-b022-44dd-b2c5-6ee2039ee54b admin admin] PATH_INFO: `/v3/projects/a26410607fd949f79355f979e90e1b23/users/f5d33ac7c0434e7cb080e5e5fb9f4e12/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.826703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 780/1570] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:33 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:33.830248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf418d7-b022-44dd-b2c5-6ee2039ee54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26410607fd949f79355f979e90e1b23, user_id=f5d33ac7c0434e7cb080e5e5fb9f4e12, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.830488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b94958-5836-4469-b51b-b807aa0fa585 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.834950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf418d7-b022-44dd-b2c5-6ee2039ee54b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.854758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 791/1571] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:33 2022] PUT /identity/v3/projects/a26410607fd949f79355f979e90e1b23/users/f5d33ac7c0434e7cb080e5e5fb9f4e12/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:33.855702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf418d7-b022-44dd-b2c5-6ee2039ee54b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.857379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b94958-5836-4469-b51b-b807aa0fa585 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.864227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:33.920412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a126f-043f-4992-8efc-d3ad1af93527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.921151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a126f-043f-4992-8efc-d3ad1af93527 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.921609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a126f-043f-4992-8efc-d3ad1af93527 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.922069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a126f-043f-4992-8efc-d3ad1af93527 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.922692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf418d7-b022-44dd-b2c5-6ee2039ee54b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.924444 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a126f-043f-4992-8efc-d3ad1af93527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.926799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a126f-043f-4992-8efc-d3ad1af93527 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.927901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e66e8c-7ecb-4307-82f9-5afa876e142f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.928624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e66e8c-7ecb-4307-82f9-5afa876e142f admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.929048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e66e8c-7ecb-4307-82f9-5afa876e142f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.929433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e66e8c-7ecb-4307-82f9-5afa876e142f admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.930324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e66e8c-7ecb-4307-82f9-5afa876e142f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.932067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e66e8c-7ecb-4307-82f9-5afa876e142f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.939148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 781/1572] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:33.943061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a126f-043f-4992-8efc-d3ad1af93527 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.943409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 792/1573] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:33.948337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e66e8c-7ecb-4307-82f9-5afa876e142f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:33.954656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a126f-043f-4992-8efc-d3ad1af93527 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.959211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e66e8c-7ecb-4307-82f9-5afa876e142f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:33.960160 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b3d24-b3d7-45f9-a80c-fea08f8bd8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.960966 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b3d24-b3d7-45f9-a80c-fea08f8bd8ae admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.961449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b3d24-b3d7-45f9-a80c-fea08f8bd8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.961796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b3d24-b3d7-45f9-a80c-fea08f8bd8ae admin admin] PATH_INFO: `/v3/projects/9708996875cd45ca885b249b335a24fe/users/47fd2100eb304efa80276f2d600689af/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.964623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac66e55-744c-4d07-900a-6a87dbf26846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:33.965905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66e55-744c-4d07-900a-6a87dbf26846 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:33.966214 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66e55-744c-4d07-900a-6a87dbf26846 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:33.966525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66e55-744c-4d07-900a-6a87dbf26846 admin admin] PATH_INFO: `/v3/projects/a26410607fd949f79355f979e90e1b23/users/f5d33ac7c0434e7cb080e5e5fb9f4e12/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:33.968477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac66e55-744c-4d07-900a-6a87dbf26846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26410607fd949f79355f979e90e1b23, user_id=f5d33ac7c0434e7cb080e5e5fb9f4e12, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.971326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac66e55-744c-4d07-900a-6a87dbf26846 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.979962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b3d24-b3d7-45f9-a80c-fea08f8bd8ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9708996875cd45ca885b249b335a24fe, user_id=47fd2100eb304efa80276f2d600689af, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:33.983595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b3d24-b3d7-45f9-a80c-fea08f8bd8ae admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:33.996664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 793/1574] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:33 2022] PUT /identity/v3/projects/a26410607fd949f79355f979e90e1b23/users/f5d33ac7c0434e7cb080e5e5fb9f4e12/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:34.003264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 782/1575] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:33 2022] PUT /identity/v3/projects/9708996875cd45ca885b249b335a24fe/users/47fd2100eb304efa80276f2d600689af/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:34.010736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207e258-708b-48fd-8954-c20f78e69a61 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:34.011021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207e258-708b-48fd-8954-c20f78e69a61 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:34.011357 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207e258-708b-48fd-8954-c20f78e69a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:34.013452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b3d24-b3d7-45f9-a80c-fea08f8bd8ae admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:34.058420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d207e258-708b-48fd-8954-c20f78e69a61 None None] MFA Rules not processed for user `f5d33ac7c0434e7cb080e5e5fb9f4e12`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:34.059758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d207e258-708b-48fd-8954-c20f78e69a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:34.095324 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b3d24-b3d7-45f9-a80c-fea08f8bd8ae admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:34.115882 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea7014-602d-4d66-82f7-39e3c6e9dcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:34.116721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea7014-602d-4d66-82f7-39e3c6e9dcda admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:34.117155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea7014-602d-4d66-82f7-39e3c6e9dcda admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:34.117498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea7014-602d-4d66-82f7-39e3c6e9dcda admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:34.118224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea7014-602d-4d66-82f7-39e3c6e9dcda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:34.120777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea7014-602d-4d66-82f7-39e3c6e9dcda admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:34.131355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 783/1576] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:34.135166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 794/1577] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:34.136016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea7014-602d-4d66-82f7-39e3c6e9dcda admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:34.160159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea7014-602d-4d66-82f7-39e3c6e9dcda admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:34.171052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298cc4f-cbcd-4ebc-b121-948cc592a1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:34.171796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298cc4f-cbcd-4ebc-b121-948cc592a1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:34.172133 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298cc4f-cbcd-4ebc-b121-948cc592a1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:34.172889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298cc4f-cbcd-4ebc-b121-948cc592a1f8 admin admin] PATH_INFO: `/v3/projects/9708996875cd45ca885b249b335a24fe/users/47fd2100eb304efa80276f2d600689af/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:34.174919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298cc4f-cbcd-4ebc-b121-948cc592a1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9708996875cd45ca885b249b335a24fe, user_id=47fd2100eb304efa80276f2d600689af, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:34.178537 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298cc4f-cbcd-4ebc-b121-948cc592a1f8 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:34.203901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 784/1578] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:34 2022] PUT /identity/v3/projects/9708996875cd45ca885b249b335a24fe/users/47fd2100eb304efa80276f2d600689af/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:34.211943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658474b-0dc1-44bf-94a1-eafe109978f8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:34.212550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658474b-0dc1-44bf-94a1-eafe109978f8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:34.213124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658474b-0dc1-44bf-94a1-eafe109978f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:34.265132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:34.284181 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-2658474b-0dc1-44bf-94a1-eafe109978f8 None None] MFA Rules not processed for user `47fd2100eb304efa80276f2d600689af`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:34.286216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2658474b-0dc1-44bf-94a1-eafe109978f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:34.409908 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 795/1579] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:34.417414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298cc4f-cbcd-4ebc-b121-948cc592a1f8 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:34.419334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9298cc4f-cbcd-4ebc-b121-948cc592a1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:34.481489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0343dd2-4231-4544-b5d6-04daf21fa9e5 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:34.482557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0343dd2-4231-4544-b5d6-04daf21fa9e5 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:34.482967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0343dd2-4231-4544-b5d6-04daf21fa9e5 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:34.485716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 796/1580] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 14:41:34.514913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2c3d3-c152-4659-b23c-95c3c1e19bc0 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:34.515423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2c3d3-c152-4659-b23c-95c3c1e19bc0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:34.515813 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2c3d3-c152-4659-b23c-95c3c1e19bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:34.549321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298cc4f-cbcd-4ebc-b121-948cc592a1f8 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:34.549321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ed33d-7d4f-4137-b40b-4a3e12039e3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:34.549321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ed33d-7d4f-4137-b40b-4a3e12039e3a service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:34.549321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ed33d-7d4f-4137-b40b-4a3e12039e3a service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:34.549784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ed33d-7d4f-4137-b40b-4a3e12039e3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:34.557468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-85d2c3d3-c152-4659-b23c-95c3c1e19bc0 None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:34.561748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-85d2c3d3-c152-4659-b23c-95c3c1e19bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:34.570096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:34.654164 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ed33d-7d4f-4137-b40b-4a3e12039e3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:34.658707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ed33d-7d4f-4137-b40b-4a3e12039e3a service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:34.698450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 785/1581] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:34.793390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 797/1582] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:34.810562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ed33d-7d4f-4137-b40b-4a3e12039e3a service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:34.813070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-145ed33d-7d4f-4137-b40b-4a3e12039e3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:34.845161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ed33d-7d4f-4137-b40b-4a3e12039e3a service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:34.857187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e0828-86c2-47eb-9548-542f6271e621 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:34.857187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e0828-86c2-47eb-9548-542f6271e621 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:34.857187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e0828-86c2-47eb-9548-542f6271e621 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:34.857187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e0828-86c2-47eb-9548-542f6271e621 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:34.866839 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2c3d3-c152-4659-b23c-95c3c1e19bc0 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:34.868430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-85d2c3d3-c152-4659-b23c-95c3c1e19bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:34.885576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e0828-86c2-47eb-9548-542f6271e621 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:34.889424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e0828-86c2-47eb-9548-542f6271e621 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:34.890774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2c3d3-c152-4659-b23c-95c3c1e19bc0 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:34.897412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5ffc4-785c-4b1e-b4e3-8f110ad3f15c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:34.898186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5ffc4-785c-4b1e-b4e3-8f110ad3f15c service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:34.898578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5ffc4-785c-4b1e-b4e3-8f110ad3f15c service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:34.898961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5ffc4-785c-4b1e-b4e3-8f110ad3f15c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:34.902603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-32c5ffc4-785c-4b1e-b4e3-8f110ad3f15c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:34.920171 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 786/1583] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:35.078456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5ffc4-785c-4b1e-b4e3-8f110ad3f15c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:35.079916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5ffc4-785c-4b1e-b4e3-8f110ad3f15c service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:35.101135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 798/1584] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:37.363951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e0828-86c2-47eb-9548-542f6271e621 service nova] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:37.365432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2c8e0828-86c2-47eb-9548-542f6271e621 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:37.406064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5ffc4-785c-4b1e-b4e3-8f110ad3f15c service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:37.415537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5ffc4-785c-4b1e-b4e3-8f110ad3f15c service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:37.420729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244f3a1-a94d-4ad4-96a6-51a05165ccf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:37.421342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244f3a1-a94d-4ad4-96a6-51a05165ccf7 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:37.421626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244f3a1-a94d-4ad4-96a6-51a05165ccf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:37.422457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244f3a1-a94d-4ad4-96a6-51a05165ccf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:37.452842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:37.464405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e0828-86c2-47eb-9548-542f6271e621 service nova] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:37.470323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:37.470974 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:37.471666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:37.471956 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] PATH_INFO: `/v3/users/b7993ef186d449b4ad933bdfa900635e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:37.475002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7993ef186d449b4ad933bdfa900635e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:37.475911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:37.500193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244f3a1-a94d-4ad4-96a6-51a05165ccf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:37.501680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244f3a1-a94d-4ad4-96a6-51a05165ccf7 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:37.513421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 799/1585] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:37.538272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7993ef186d449b4ad933bdfa900635e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:37.548058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7993ef186d449b4ad933bdfa900635e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:37.548871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7993ef186d449b4ad933bdfa900635e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:37.549901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7993ef186d449b4ad933bdfa900635e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:37.560631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7993ef186d449b4ad933bdfa900635e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:37.585949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 787/1586] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:37 2022] DELETE /identity/v3/users/b7993ef186d449b4ad933bdfa900635e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:37.591482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb1dba-5e3d-4ab0-94cf-4f6bdfe887db None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:37.591886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb1dba-5e3d-4ab0-94cf-4f6bdfe887db None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:37.592283 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb1dba-5e3d-4ab0-94cf-4f6bdfe887db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:37.612648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d0bb1dba-5e3d-4ab0-94cf-4f6bdfe887db None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:37.613898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d0bb1dba-5e3d-4ab0-94cf-4f6bdfe887db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:37.683287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 800/1587] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:37.693315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:37.694738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:37.745843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:37.786096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a829-be16-4b0b-9845-6b96c0691656 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:37.790540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e9130-6243-41e3-b0c5-45f3f4809c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:37.791706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e9130-6243-41e3-b0c5-45f3f4809c83 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:37.792082 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e9130-6243-41e3-b0c5-45f3f4809c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:37.792331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e9130-6243-41e3-b0c5-45f3f4809c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:37.802943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e9130-6243-41e3-b0c5-45f3f4809c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:37.804425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e9130-6243-41e3-b0c5-45f3f4809c83 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:37.816235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 788/1588] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:37.908689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb1dba-5e3d-4ab0-94cf-4f6bdfe887db None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:37.910765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d0bb1dba-5e3d-4ab0-94cf-4f6bdfe887db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:37.992668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb1dba-5e3d-4ab0-94cf-4f6bdfe887db None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:37.997793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa833b47-f403-4ca5-b68a-6639411a1fb8 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5d33ac7c0434e7cb080e5e5fb9f4e12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a26410607fd949f79355f979e90e1b23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:37.999317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa833b47-f403-4ca5-b68a-6639411a1fb8 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:37.999610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa833b47-f403-4ca5-b68a-6639411a1fb8 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:37.999893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa833b47-f403-4ca5-b68a-6639411a1fb8 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:38.000592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa833b47-f403-4ca5-b68a-6639411a1fb8 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:38.000905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa833b47-f403-4ca5-b68a-6639411a1fb8 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:38.002716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa833b47-f403-4ca5-b68a-6639411a1fb8 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:38.016203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 801/1589] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:38.025151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e9130-6243-41e3-b0c5-45f3f4809c83 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:38.048578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e9130-6243-41e3-b0c5-45f3f4809c83 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:38.068617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b368cd-9353-4101-8b54-c08df4da799f tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5d33ac7c0434e7cb080e5e5fb9f4e12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a26410607fd949f79355f979e90e1b23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:38.068617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b368cd-9353-4101-8b54-c08df4da799f tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:38.068617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b368cd-9353-4101-8b54-c08df4da799f tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:38.068939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b368cd-9353-4101-8b54-c08df4da799f tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:38.068939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b368cd-9353-4101-8b54-c08df4da799f tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:38.068939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b368cd-9353-4101-8b54-c08df4da799f tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:38.093185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 789/1590] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:38 2022] POST /identity/v3/projects => generated 354 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:38.097168 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa833b47-f403-4ca5-b68a-6639411a1fb8 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:38.109246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa833b47-f403-4ca5-b68a-6639411a1fb8 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:38.170580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:38.177689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5d33ac7c0434e7cb080e5e5fb9f4e12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a26410607fd949f79355f979e90e1b23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:38.178306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:38.178595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:38.178874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:38.179527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:38.179839 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:38.186029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:38.198225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 802/1591] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:38.238375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b368cd-9353-4101-8b54-c08df4da799f tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:38.249588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b368cd-9353-4101-8b54-c08df4da799f tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:38.254642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5d33ac7c0434e7cb080e5e5fb9f4e12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a26410607fd949f79355f979e90e1b23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:38.255664 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:38.255772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:38.256014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] PATH_INFO: `/v3/projects/1f2ac82833e54f2aa35c89eba81379fe` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:38.257566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1f2ac82833e54f2aa35c89eba81379fe)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:38.259320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:38.310114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2ac82833e54f2aa35c89eba81379fe', 'request_id': 'req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.316736 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2ac82833e54f2aa35c89eba81379fe', 'request_id': 'req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.326110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2ac82833e54f2aa35c89eba81379fe', 'request_id': 'req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.340246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] The token cache is being invalidate because project 1f2ac82833e54f2aa35c89eba81379fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:38.340246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.344379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.346276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 790/1592] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:38 2022] DELETE /identity/v3/projects/1f2ac82833e54f2aa35c89eba81379fe => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:38.371234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:38.373113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:38.459865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eab70c-bcc4-429f-a039-e09047f8ffa6 tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:38.550915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:38.551944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:38.552674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:38.553313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] PATH_INFO: `/v3/projects/7df8b13bbfee41be9c8b84463bbc084c` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:38.555720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df8b13bbfee41be9c8b84463bbc084c)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:38.559617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:38.566480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:38.568846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:38.587842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9ca75-1bdd-45b0-ae8e-b6835840a72a tempest-VolumeQuotaClassesTest-627867018 tempest-VolumeQuotaClassesTest-627867018-project-admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:38.596140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:38.597009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:38.597287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:38.597555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] PATH_INFO: `/v3/users/31129b01036f49fdb1a2f20af970db9b` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:38.599377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31129b01036f49fdb1a2f20af970db9b)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:38.602486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:38.633676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df8b13bbfee41be9c8b84463bbc084c', 'request_id': 'req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.642119 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df8b13bbfee41be9c8b84463bbc084c', 'request_id': 'req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.656688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31129b01036f49fdb1a2f20af970db9b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.660425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df8b13bbfee41be9c8b84463bbc084c', 'request_id': 'req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.664087 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31129b01036f49fdb1a2f20af970db9b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.664769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31129b01036f49fdb1a2f20af970db9b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.665367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31129b01036f49fdb1a2f20af970db9b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.673525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df1962f9-d149-4749-aae3-eac11acc6da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31129b01036f49fdb1a2f20af970db9b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.674258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] The token cache is being invalidate because project 7df8b13bbfee41be9c8b84463bbc084c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:38.675032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.675873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.677467 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 803/1593] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:38 2022] DELETE /identity/v3/projects/7df8b13bbfee41be9c8b84463bbc084c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:38.681242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:38.683631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:38.688588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:38.697046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 791/1594] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:38 2022] DELETE /identity/v3/users/31129b01036f49fdb1a2f20af970db9b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:38.702698 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244868b8-50f8-4b9d-b93b-dd0e04ae4341 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:38.702983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244868b8-50f8-4b9d-b93b-dd0e04ae4341 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:38.703258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244868b8-50f8-4b9d-b93b-dd0e04ae4341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:38.718822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:38.725314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-244868b8-50f8-4b9d-b93b-dd0e04ae4341 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:38.726472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-244868b8-50f8-4b9d-b93b-dd0e04ae4341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:38.760303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f3daa-3d55-45d8-9abb-af57deb6e3bf admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:38.789702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 792/1595] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:38.800359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244868b8-50f8-4b9d-b93b-dd0e04ae4341 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:38.801818 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-244868b8-50f8-4b9d-b93b-dd0e04ae4341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:38.824584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:38.825230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:38.825512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:38.825788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] PATH_INFO: `/v3/users/6f775b0eb5a64d819f2195edd0c6865b` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:38.826931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f775b0eb5a64d819f2195edd0c6865b)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:38.828908 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:38.862084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244868b8-50f8-4b9d-b93b-dd0e04ae4341 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:38.866058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023aab72-7cf7-4cd6-b801-d216ad54198c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:38.866670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023aab72-7cf7-4cd6-b801-d216ad54198c service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:38.866950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023aab72-7cf7-4cd6-b801-d216ad54198c service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:38.867230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023aab72-7cf7-4cd6-b801-d216ad54198c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:38.876653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023aab72-7cf7-4cd6-b801-d216ad54198c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:38.878124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023aab72-7cf7-4cd6-b801-d216ad54198c service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:38.881775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f775b0eb5a64d819f2195edd0c6865b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.888738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 793/1596] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:38.891582 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f775b0eb5a64d819f2195edd0c6865b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.892304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f775b0eb5a64d819f2195edd0c6865b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.893012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f775b0eb5a64d819f2195edd0c6865b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.902937 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f775b0eb5a64d819f2195edd0c6865b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:38.922012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 804/1597] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:38 2022] DELETE /identity/v3/users/6f775b0eb5a64d819f2195edd0c6865b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:39.320162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023aab72-7cf7-4cd6-b801-d216ad54198c service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:39.333562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023aab72-7cf7-4cd6-b801-d216ad54198c service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:39.412173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:39.412921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:39.413209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:39.413488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] PATH_INFO: `/v3/projects/ef03e0cd57b8443790002e1e5e4e9aa9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:39.414738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef03e0cd57b8443790002e1e5e4e9aa9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:39.417304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:39.460564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:39.464854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:39.473037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:39.479823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef03e0cd57b8443790002e1e5e4e9aa9', 'request_id': 'req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.486811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f3ac1-3e25-4702-b24d-0eaa16ed66a4 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:39.490587 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef03e0cd57b8443790002e1e5e4e9aa9', 'request_id': 'req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.504292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef03e0cd57b8443790002e1e5e4e9aa9', 'request_id': 'req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.522600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] The token cache is being invalidate because project ef03e0cd57b8443790002e1e5e4e9aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:39.523332 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.525007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.527844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 794/1598] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:39 2022] DELETE /identity/v3/projects/ef03e0cd57b8443790002e1e5e4e9aa9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:39.531020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:39.532764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:39.570557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:39.586953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:39.588257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:39.589109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:39.589620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] PATH_INFO: `/v3/projects/7f4f2c29f23846b7804baf6f95892907` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:39.591906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f4f2c29f23846b7804baf6f95892907)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:39.599208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:39.663800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb3f38-bd91-4594-85f8-5d38ea48b20f admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:39.669420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:39.670099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:39.670376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:39.670648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] PATH_INFO: `/v3/users/ef4f1a005c574736a9bc485470770c9a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:39.671799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef4f1a005c574736a9bc485470770c9a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:39.674576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:39.730074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f4f2c29f23846b7804baf6f95892907', 'request_id': 'req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.734004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef4f1a005c574736a9bc485470770c9a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.736804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f4f2c29f23846b7804baf6f95892907', 'request_id': 'req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.746006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef4f1a005c574736a9bc485470770c9a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.746881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef4f1a005c574736a9bc485470770c9a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.746990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f4f2c29f23846b7804baf6f95892907', 'request_id': 'req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.747556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef4f1a005c574736a9bc485470770c9a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.760662 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef4f1a005c574736a9bc485470770c9a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.762907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] The token cache is being invalidate because project 7f4f2c29f23846b7804baf6f95892907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:39.763295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.764374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:39.766499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 805/1599] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:39 2022] DELETE /identity/v3/projects/7f4f2c29f23846b7804baf6f95892907 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:39.770652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:39.772194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:39.791464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 795/1600] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:39 2022] DELETE /identity/v3/users/ef4f1a005c574736a9bc485470770c9a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:39.867264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bb32e-5e31-4225-be3a-64db5adcd9b1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:39.934560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:39.935200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:39.935484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:39.935757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] PATH_INFO: `/v3/users/f5d33ac7c0434e7cb080e5e5fb9f4e12` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:39.936949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d33ac7c0434e7cb080e5e5fb9f4e12)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:39.939070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:39.994851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:40.014120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d33ac7c0434e7cb080e5e5fb9f4e12'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.025400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d33ac7c0434e7cb080e5e5fb9f4e12'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.026836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d33ac7c0434e7cb080e5e5fb9f4e12'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.027970 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d33ac7c0434e7cb080e5e5fb9f4e12'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.046100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fe7fdcff-2b1d-4d44-a683-87f5aba7f833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d33ac7c0434e7cb080e5e5fb9f4e12'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.132152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 806/1601] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:39 2022] DELETE /identity/v3/users/f5d33ac7c0434e7cb080e5e5fb9f4e12 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:40.135888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:40.142609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:40.158119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd3326-94d6-47bb-96c8-ba68d8d003d6 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:40.229539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:40.245051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:40.246039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:40.246543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:40.246987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] PATH_INFO: `/v3/projects/ce1053f1a9e54f6aa9be2a49a1064e78` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:40.248334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1053f1a9e54f6aa9be2a49a1064e78)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:40.251110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:40.336190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1053f1a9e54f6aa9be2a49a1064e78', 'request_id': 'req-23fd4c78-1368-4a05-9c09-0c5ce3931d72'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.346007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1053f1a9e54f6aa9be2a49a1064e78', 'request_id': 'req-23fd4c78-1368-4a05-9c09-0c5ce3931d72'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.359137 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1053f1a9e54f6aa9be2a49a1064e78', 'request_id': 'req-23fd4c78-1368-4a05-9c09-0c5ce3931d72'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.372602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] The token cache is being invalidate because project ce1053f1a9e54f6aa9be2a49a1064e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:40.372897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.373553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.379602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 796/1602] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:40 2022] DELETE /identity/v3/projects/ce1053f1a9e54f6aa9be2a49a1064e78 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:40.381277 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc48d6-cc4a-4dce-a0a1-49b7bdc20019 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:40.381587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc48d6-cc4a-4dce-a0a1-49b7bdc20019 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:40.381853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc48d6-cc4a-4dce-a0a1-49b7bdc20019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:40.404417 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-eefc48d6-cc4a-4dce-a0a1-49b7bdc20019 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:40.405712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-eefc48d6-cc4a-4dce-a0a1-49b7bdc20019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:40.502819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:40.504502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:40.553074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 807/1603] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:40.564426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d602b-c5d8-40a7-a3c8-2a8ac06aa943 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:40.564667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d602b-c5d8-40a7-a3c8-2a8ac06aa943 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:40.564944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d602b-c5d8-40a7-a3c8-2a8ac06aa943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:40.603096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-548d602b-c5d8-40a7-a3c8-2a8ac06aa943 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:40.604868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-548d602b-c5d8-40a7-a3c8-2a8ac06aa943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:40.618498 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 808/1604] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:41:40.620782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d602b-c5d8-40a7-a3c8-2a8ac06aa943 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:40.630878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d602b-c5d8-40a7-a3c8-2a8ac06aa943 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:40.642040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56f3aa-955e-494f-9e84-69a787ecb56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:40.642692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56f3aa-955e-494f-9e84-69a787ecb56f admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:40.642973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56f3aa-955e-494f-9e84-69a787ecb56f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:40.643255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56f3aa-955e-494f-9e84-69a787ecb56f admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:40.644496 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56f3aa-955e-494f-9e84-69a787ecb56f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:40.644496 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56f3aa-955e-494f-9e84-69a787ecb56f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:40.645909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56f3aa-955e-494f-9e84-69a787ecb56f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:40.650861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd4c78-1368-4a05-9c09-0c5ce3931d72 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:40.657542 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:40.660054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:40.660724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:40.661765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] PATH_INFO: `/v3/projects/a26410607fd949f79355f979e90e1b23` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:40.663319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26410607fd949f79355f979e90e1b23)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:40.666488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:40.681254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 809/1605] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:40.686258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56f3aa-955e-494f-9e84-69a787ecb56f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:40.706351 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56f3aa-955e-494f-9e84-69a787ecb56f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:40.716584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a54e2b-dbe6-4cbe-aae7-41a0bc4e0219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:40.717334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54e2b-dbe6-4cbe-aae7-41a0bc4e0219 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:40.717617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54e2b-dbe6-4cbe-aae7-41a0bc4e0219 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:40.717897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54e2b-dbe6-4cbe-aae7-41a0bc4e0219 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:40.720303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a54e2b-dbe6-4cbe-aae7-41a0bc4e0219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:40.723157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a54e2b-dbe6-4cbe-aae7-41a0bc4e0219 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:40.768458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26410607fd949f79355f979e90e1b23', 'request_id': 'req-b756cf22-d5c5-405a-a344-11f5bc41d687'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.780932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26410607fd949f79355f979e90e1b23', 'request_id': 'req-b756cf22-d5c5-405a-a344-11f5bc41d687'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.795003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26410607fd949f79355f979e90e1b23', 'request_id': 'req-b756cf22-d5c5-405a-a344-11f5bc41d687'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.798278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 810/1606] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:40 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:40.808388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a54e2b-dbe6-4cbe-aae7-41a0bc4e0219 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:40.813815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] The token cache is being invalidate because project a26410607fd949f79355f979e90e1b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:40.813815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.813815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:40.814968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 797/1607] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:40 2022] DELETE /identity/v3/projects/a26410607fd949f79355f979e90e1b23 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:40.815483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-73a54e2b-dbe6-4cbe-aae7-41a0bc4e0219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:40.815837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:40.817504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:40.832673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:40.891027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a54e2b-dbe6-4cbe-aae7-41a0bc4e0219 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:40.896550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28544c78-05e5-4248-95bb-6f33defb7764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:40.897188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28544c78-05e5-4248-95bb-6f33defb7764 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:40.897292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756cf22-d5c5-405a-a344-11f5bc41d687 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:40.897599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28544c78-05e5-4248-95bb-6f33defb7764 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:40.898131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28544c78-05e5-4248-95bb-6f33defb7764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:40.902085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-28544c78-05e5-4248-95bb-6f33defb7764 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:40.941000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:40.967420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1e67b-295c-47fd-93de-fff4fcb56206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:40.968728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1e67b-295c-47fd-93de-fff4fcb56206 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:40.969089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1e67b-295c-47fd-93de-fff4fcb56206 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:40.969440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1e67b-295c-47fd-93de-fff4fcb56206 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:40.970158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1e67b-295c-47fd-93de-fff4fcb56206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:40.972200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1e67b-295c-47fd-93de-fff4fcb56206 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:40.999319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 798/1608] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:40 2022] POST /identity/v3/users => generated 305 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:40.999319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c6ace-fbf0-454f-b1fa-5f4f020c763f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:40.999319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c6ace-fbf0-454f-b1fa-5f4f020c763f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:40.999319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c6ace-fbf0-454f-b1fa-5f4f020c763f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.012540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28544c78-05e5-4248-95bb-6f33defb7764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.015309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28544c78-05e5-4248-95bb-6f33defb7764 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.030068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-274c6ace-fbf0-454f-b1fa-5f4f020c763f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:41.031327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-274c6ace-fbf0-454f-b1fa-5f4f020c763f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:41.040091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 799/1609] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:41.041899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c6ace-fbf0-454f-b1fa-5f4f020c763f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.047341 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 811/1610] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:41.051480 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c6ace-fbf0-454f-b1fa-5f4f020c763f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.053994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434104e-09b4-40d0-8c36-b3b6a66db62e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.054352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434104e-09b4-40d0-8c36-b3b6a66db62e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.054699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434104e-09b4-40d0-8c36-b3b6a66db62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.057343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d847b-c688-4ab5-bb66-be02a22eacce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.058082 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d847b-c688-4ab5-bb66-be02a22eacce admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.058366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d847b-c688-4ab5-bb66-be02a22eacce admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.058644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d847b-c688-4ab5-bb66-be02a22eacce admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.059291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d847b-c688-4ab5-bb66-be02a22eacce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.060973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d847b-c688-4ab5-bb66-be02a22eacce admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.074505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 800/1611] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:41.075686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d847b-c688-4ab5-bb66-be02a22eacce admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.087473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-8434104e-09b4-40d0-8c36-b3b6a66db62e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:41.089443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8434104e-09b4-40d0-8c36-b3b6a66db62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:41.092628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d847b-c688-4ab5-bb66-be02a22eacce admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.098602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 812/1612] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:41.103010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434104e-09b4-40d0-8c36-b3b6a66db62e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.113524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d35311-82fe-4891-bad0-a71455605ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.113726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434104e-09b4-40d0-8c36-b3b6a66db62e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.114248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d35311-82fe-4891-bad0-a71455605ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.114594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d35311-82fe-4891-bad0-a71455605ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.114939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d35311-82fe-4891-bad0-a71455605ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.115641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d35311-82fe-4891-bad0-a71455605ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.117760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d35311-82fe-4891-bad0-a71455605ef0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.118571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feccc-faa8-4386-9971-1a0a04f057a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.121825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feccc-faa8-4386-9971-1a0a04f057a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.121825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feccc-faa8-4386-9971-1a0a04f057a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.121825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feccc-faa8-4386-9971-1a0a04f057a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.124873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578feccc-faa8-4386-9971-1a0a04f057a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:41.125365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578feccc-faa8-4386-9971-1a0a04f057a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.128041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578feccc-faa8-4386-9971-1a0a04f057a1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.130867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 801/1613] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:41.135032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d35311-82fe-4891-bad0-a71455605ef0 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.143737 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 813/1614] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:41:41.146282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d35311-82fe-4891-bad0-a71455605ef0 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.148480 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feccc-faa8-4386-9971-1a0a04f057a1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.152757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594311c-dd43-49eb-a31e-9847513b8f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.153780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594311c-dd43-49eb-a31e-9847513b8f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.154427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594311c-dd43-49eb-a31e-9847513b8f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.154801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594311c-dd43-49eb-a31e-9847513b8f98 admin admin] PATH_INFO: `/v3/projects/5f2d05ef79b74db9b589af9ca0a63f1a/users/243452abe98040de9ee11ddf513c87c3/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.164080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feccc-faa8-4386-9971-1a0a04f057a1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.170754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328964b-04c2-422e-9311-8f4f78ab2ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.171807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328964b-04c2-422e-9311-8f4f78ab2ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.172357 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328964b-04c2-422e-9311-8f4f78ab2ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.172943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328964b-04c2-422e-9311-8f4f78ab2ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.173817 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328964b-04c2-422e-9311-8f4f78ab2ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.176474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328964b-04c2-422e-9311-8f4f78ab2ef1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.198314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594311c-dd43-49eb-a31e-9847513b8f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2d05ef79b74db9b589af9ca0a63f1a, user_id=243452abe98040de9ee11ddf513c87c3, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.201963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594311c-dd43-49eb-a31e-9847513b8f98 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.211366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 814/1615] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:41 2022] POST /identity/v3/projects => generated 391 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:41.216507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328964b-04c2-422e-9311-8f4f78ab2ef1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.229096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 802/1616] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:41 2022] PUT /identity/v3/projects/5f2d05ef79b74db9b589af9ca0a63f1a/users/243452abe98040de9ee11ddf513c87c3/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:41.232474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4c549-0fc3-4668-a173-c735fa9003b5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.232767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4c549-0fc3-4668-a173-c735fa9003b5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.233049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4c549-0fc3-4668-a173-c735fa9003b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.272468 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ffc4c549-0fc3-4668-a173-c735fa9003b5 None None] MFA Rules not processed for user `243452abe98040de9ee11ddf513c87c3`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:41.273643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ffc4c549-0fc3-4668-a173-c735fa9003b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:41.312608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328964b-04c2-422e-9311-8f4f78ab2ef1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.341397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:41.380958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 803/1617] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:41 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:41.384145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4c549-0fc3-4668-a173-c735fa9003b5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.395210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a104b47-89b1-4cec-bd86-7e1c59fec1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.396418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a104b47-89b1-4cec-bd86-7e1c59fec1fd admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.396816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a104b47-89b1-4cec-bd86-7e1c59fec1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.397783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a104b47-89b1-4cec-bd86-7e1c59fec1fd admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.398746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a104b47-89b1-4cec-bd86-7e1c59fec1fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.401512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a104b47-89b1-4cec-bd86-7e1c59fec1fd admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.430086 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 815/1618] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:41 2022] POST /identity/v3/users => generated 296 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:41.434019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a104b47-89b1-4cec-bd86-7e1c59fec1fd admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.449556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a104b47-89b1-4cec-bd86-7e1c59fec1fd admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.458304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab58b1-0469-466a-b0ce-1a8f4d7c23ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.459186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab58b1-0469-466a-b0ce-1a8f4d7c23ec admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.459534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab58b1-0469-466a-b0ce-1a8f4d7c23ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.459900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab58b1-0469-466a-b0ce-1a8f4d7c23ec admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.460649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab58b1-0469-466a-b0ce-1a8f4d7c23ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.462365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab58b1-0469-466a-b0ce-1a8f4d7c23ec admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.473777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 816/1619] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:41.477850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab58b1-0469-466a-b0ce-1a8f4d7c23ec admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.478456 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4c549-0fc3-4668-a173-c735fa9003b5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.483006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:41.485201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acec26-1896-480f-beec-14e6b0d62cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.486073 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acec26-1896-480f-beec-14e6b0d62cfc admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.486363 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acec26-1896-480f-beec-14e6b0d62cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.486637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acec26-1896-480f-beec-14e6b0d62cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.487765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acec26-1896-480f-beec-14e6b0d62cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.488378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab58b1-0469-466a-b0ce-1a8f4d7c23ec admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.489873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acec26-1896-480f-beec-14e6b0d62cfc admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.494910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f72cf1-fcbe-4d82-b6c2-6cdb5d395502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.495651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f72cf1-fcbe-4d82-b6c2-6cdb5d395502 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.495945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f72cf1-fcbe-4d82-b6c2-6cdb5d395502 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.496248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f72cf1-fcbe-4d82-b6c2-6cdb5d395502 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.497068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f72cf1-fcbe-4d82-b6c2-6cdb5d395502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.498864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f72cf1-fcbe-4d82-b6c2-6cdb5d395502 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.512023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 817/1620] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:41.528092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f72cf1-fcbe-4d82-b6c2-6cdb5d395502 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.539618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f72cf1-fcbe-4d82-b6c2-6cdb5d395502 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.545925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2951d3-6176-4e6f-a244-80452cc78251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.546606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 804/1621] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:41 2022] POST /identity/v3/projects => generated 409 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:41.546721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2951d3-6176-4e6f-a244-80452cc78251 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.547208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2951d3-6176-4e6f-a244-80452cc78251 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.547492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2951d3-6176-4e6f-a244-80452cc78251 admin admin] PATH_INFO: `/v3/projects/6328839e04cb4b1fae432022e904e98c/users/38bfae8a463a4a329a927a14615aeac7/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.550754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acec26-1896-480f-beec-14e6b0d62cfc admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.565660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2951d3-6176-4e6f-a244-80452cc78251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6328839e04cb4b1fae432022e904e98c, user_id=38bfae8a463a4a329a927a14615aeac7, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.568413 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2951d3-6176-4e6f-a244-80452cc78251 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.583470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 818/1622] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:41 2022] PUT /identity/v3/projects/6328839e04cb4b1fae432022e904e98c/users/38bfae8a463a4a329a927a14615aeac7/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:41.594510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1b533-d0dc-437a-ad2e-b89be27397fb None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.594793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1b533-d0dc-437a-ad2e-b89be27397fb None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.595066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1b533-d0dc-437a-ad2e-b89be27397fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.622181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acec26-1896-480f-beec-14e6b0d62cfc admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.623949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-5ae1b533-d0dc-437a-ad2e-b89be27397fb None None] MFA Rules not processed for user `38bfae8a463a4a329a927a14615aeac7`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:41.625264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5ae1b533-d0dc-437a-ad2e-b89be27397fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:41.681248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a801f-54eb-4ca0-9c93-0855a086e3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.681911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a801f-54eb-4ca0-9c93-0855a086e3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.682176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a801f-54eb-4ca0-9c93-0855a086e3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.682462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a801f-54eb-4ca0-9c93-0855a086e3f3 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.683097 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a801f-54eb-4ca0-9c93-0855a086e3f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.685117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a801f-54eb-4ca0-9c93-0855a086e3f3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.695626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 819/1623] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:41:41.701349 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1b533-d0dc-437a-ad2e-b89be27397fb None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.708195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 805/1624] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:41 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:41.711555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a801f-54eb-4ca0-9c93-0855a086e3f3 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.712872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1b533-d0dc-437a-ad2e-b89be27397fb None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.717399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3fb5d-acfa-4ba7-b116-30023a33edd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.717999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3fb5d-acfa-4ba7-b116-30023a33edd2 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.718276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3fb5d-acfa-4ba7-b116-30023a33edd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.718545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3fb5d-acfa-4ba7-b116-30023a33edd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.719261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3fb5d-acfa-4ba7-b116-30023a33edd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.721288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3fb5d-acfa-4ba7-b116-30023a33edd2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.721410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a801f-54eb-4ca0-9c93-0855a086e3f3 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.725868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1438b4-58d9-4b9a-9882-5ed6834d6bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.726571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1438b4-58d9-4b9a-9882-5ed6834d6bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.726888 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1438b4-58d9-4b9a-9882-5ed6834d6bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.727178 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1438b4-58d9-4b9a-9882-5ed6834d6bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.728120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1438b4-58d9-4b9a-9882-5ed6834d6bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.730282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1438b4-58d9-4b9a-9882-5ed6834d6bf7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.732528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:41.742140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 806/1625] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:41.746361 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1438b4-58d9-4b9a-9882-5ed6834d6bf7 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.756363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 820/1626] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:41 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:41.756498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1438b4-58d9-4b9a-9882-5ed6834d6bf7 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.757505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3fb5d-acfa-4ba7-b116-30023a33edd2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.819811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a8254-df59-43bf-a123-5a43ea5e5b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.820699 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a8254-df59-43bf-a123-5a43ea5e5b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.820929 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a8254-df59-43bf-a123-5a43ea5e5b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.821211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a8254-df59-43bf-a123-5a43ea5e5b35 admin admin] PATH_INFO: `/v3/projects/75081191b01b4456bdab2e43ff64f547/users/af937f8306cc4874b6c7dbde68b0ccb2/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.827517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3fb5d-acfa-4ba7-b116-30023a33edd2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.833957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45225b02-af17-4afb-acf5-81662f4ac8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.834621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45225b02-af17-4afb-acf5-81662f4ac8ca admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.834907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45225b02-af17-4afb-acf5-81662f4ac8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.835180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45225b02-af17-4afb-acf5-81662f4ac8ca admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.835812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45225b02-af17-4afb-acf5-81662f4ac8ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.838859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45225b02-af17-4afb-acf5-81662f4ac8ca admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.839557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a8254-df59-43bf-a123-5a43ea5e5b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75081191b01b4456bdab2e43ff64f547, user_id=af937f8306cc4874b6c7dbde68b0ccb2, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.842788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a8254-df59-43bf-a123-5a43ea5e5b35 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.867471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 821/1627] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:41 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:41.872560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 807/1628] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:41 2022] PUT /identity/v3/projects/75081191b01b4456bdab2e43ff64f547/users/af937f8306cc4874b6c7dbde68b0ccb2/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:41.873268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45225b02-af17-4afb-acf5-81662f4ac8ca admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.873709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a8254-df59-43bf-a123-5a43ea5e5b35 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.891089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45225b02-af17-4afb-acf5-81662f4ac8ca admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.916944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:41.958339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a8254-df59-43bf-a123-5a43ea5e5b35 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:41.964865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e158f-ef98-40c0-a58e-2e3a1cadb5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.965017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642eea3-f954-4116-8065-a1bd6835d074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:41.965562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642eea3-f954-4116-8065-a1bd6835d074 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.965842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642eea3-f954-4116-8065-a1bd6835d074 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.966117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642eea3-f954-4116-8065-a1bd6835d074 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.966478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e158f-ef98-40c0-a58e-2e3a1cadb5fe admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:41.966763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e158f-ef98-40c0-a58e-2e3a1cadb5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:41.966897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642eea3-f954-4116-8065-a1bd6835d074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.967076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e158f-ef98-40c0-a58e-2e3a1cadb5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:41.967684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e158f-ef98-40c0-a58e-2e3a1cadb5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:41.968412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642eea3-f954-4116-8065-a1bd6835d074 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.969399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e158f-ef98-40c0-a58e-2e3a1cadb5fe admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:41.979636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 822/1629] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:41.983622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642eea3-f954-4116-8065-a1bd6835d074 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.984244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 808/1630] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:41.988230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e158f-ef98-40c0-a58e-2e3a1cadb5fe admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:41.998732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642eea3-f954-4116-8065-a1bd6835d074 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:42.000185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e158f-ef98-40c0-a58e-2e3a1cadb5fe admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:42.003591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6d990-2dff-495f-a265-91811aa78adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:42.004330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6d990-2dff-495f-a265-91811aa78adc admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.004706 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6d990-2dff-495f-a265-91811aa78adc admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.005049 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6d990-2dff-495f-a265-91811aa78adc admin admin] PATH_INFO: `/v3/projects/b249398b7a04483bb8ce1e5544f3dddf/users/6d05bb2586e14ef186d56ff7e4899bcb/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.007330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3fc0c-158e-49e7-95f9-21e6eeff7418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:42.008051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3fc0c-158e-49e7-95f9-21e6eeff7418 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.008444 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3fc0c-158e-49e7-95f9-21e6eeff7418 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.008727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3fc0c-158e-49e7-95f9-21e6eeff7418 admin admin] PATH_INFO: `/v3/projects/75081191b01b4456bdab2e43ff64f547/users/af937f8306cc4874b6c7dbde68b0ccb2/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.011451 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3fc0c-158e-49e7-95f9-21e6eeff7418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75081191b01b4456bdab2e43ff64f547, user_id=af937f8306cc4874b6c7dbde68b0ccb2, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:42.014393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3fc0c-158e-49e7-95f9-21e6eeff7418 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:42.023178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6d990-2dff-495f-a265-91811aa78adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b249398b7a04483bb8ce1e5544f3dddf, user_id=6d05bb2586e14ef186d56ff7e4899bcb, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:42.026531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6d990-2dff-495f-a265-91811aa78adc admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:42.027236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 809/1631] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:41 2022] PUT /identity/v3/projects/75081191b01b4456bdab2e43ff64f547/users/af937f8306cc4874b6c7dbde68b0ccb2/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:42.040551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb440a8-34ea-44fd-b435-ac0e8a46333c None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.040788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb440a8-34ea-44fd-b435-ac0e8a46333c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.041057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb440a8-34ea-44fd-b435-ac0e8a46333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.045648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 823/1632] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:41 2022] PUT /identity/v3/projects/b249398b7a04483bb8ce1e5544f3dddf/users/6d05bb2586e14ef186d56ff7e4899bcb/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:42.049477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6d990-2dff-495f-a265-91811aa78adc admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:42.094004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ceb440a8-34ea-44fd-b435-ac0e8a46333c None None] MFA Rules not processed for user `af937f8306cc4874b6c7dbde68b0ccb2`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:42.095331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ceb440a8-34ea-44fd-b435-ac0e8a46333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:42.140998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6d990-2dff-495f-a265-91811aa78adc admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:42.147246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0fac6-ccb1-4260-83c6-2499390bd307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:42.147887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0fac6-ccb1-4260-83c6-2499390bd307 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.148206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0fac6-ccb1-4260-83c6-2499390bd307 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.148526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0fac6-ccb1-4260-83c6-2499390bd307 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.149184 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0fac6-ccb1-4260-83c6-2499390bd307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:42.151436 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0fac6-ccb1-4260-83c6-2499390bd307 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:42.172448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 824/1633] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:42.177104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0fac6-ccb1-4260-83c6-2499390bd307 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:42.184433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:42.191916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0fac6-ccb1-4260-83c6-2499390bd307 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:42.198344 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e60f6-0b2c-43c6-8e09-9af2b08f4db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:42.199557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e60f6-0b2c-43c6-8e09-9af2b08f4db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.199974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e60f6-0b2c-43c6-8e09-9af2b08f4db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.200256 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e60f6-0b2c-43c6-8e09-9af2b08f4db5 admin admin] PATH_INFO: `/v3/projects/b249398b7a04483bb8ce1e5544f3dddf/users/6d05bb2586e14ef186d56ff7e4899bcb/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.203024 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e60f6-0b2c-43c6-8e09-9af2b08f4db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b249398b7a04483bb8ce1e5544f3dddf, user_id=6d05bb2586e14ef186d56ff7e4899bcb, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:42.205780 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e60f6-0b2c-43c6-8e09-9af2b08f4db5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:42.215289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 810/1634] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:42 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:42.218835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 825/1635] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:42 2022] PUT /identity/v3/projects/b249398b7a04483bb8ce1e5544f3dddf/users/6d05bb2586e14ef186d56ff7e4899bcb/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:42.230289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb440a8-34ea-44fd-b435-ac0e8a46333c None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:42.231025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedef732-718e-4713-803a-510a38dd3072 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.231580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedef732-718e-4713-803a-510a38dd3072 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.231803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ceb440a8-34ea-44fd-b435-ac0e8a46333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:42.232406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedef732-718e-4713-803a-510a38dd3072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.267495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-aedef732-718e-4713-803a-510a38dd3072 None None] MFA Rules not processed for user `6d05bb2586e14ef186d56ff7e4899bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:42.269007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-aedef732-718e-4713-803a-510a38dd3072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:42.347194 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb440a8-34ea-44fd-b435-ac0e8a46333c None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:42.382803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 826/1636] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:42 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:42.420206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302e18d-cac8-4704-95cb-0a8bd491e1a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:42.421015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302e18d-cac8-4704-95cb-0a8bd491e1a2 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.421382 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302e18d-cac8-4704-95cb-0a8bd491e1a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.421865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302e18d-cac8-4704-95cb-0a8bd491e1a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.484691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302e18d-cac8-4704-95cb-0a8bd491e1a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:42.486403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302e18d-cac8-4704-95cb-0a8bd491e1a2 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:42.496302 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 811/1637] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:42 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:42.543063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedef732-718e-4713-803a-510a38dd3072 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:42.550239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:42.554468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedef732-718e-4713-803a-510a38dd3072 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:42.559899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47d7a9-4593-4d2e-9b2c-892f696e0c43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:42.560687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47d7a9-4593-4d2e-9b2c-892f696e0c43 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.561072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47d7a9-4593-4d2e-9b2c-892f696e0c43 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.561426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47d7a9-4593-4d2e-9b2c-892f696e0c43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.574393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47d7a9-4593-4d2e-9b2c-892f696e0c43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:42.575855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47d7a9-4593-4d2e-9b2c-892f696e0c43 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:42.587442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 827/1638] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:42 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:42.722628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302e18d-cac8-4704-95cb-0a8bd491e1a2 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:42.728750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47d7a9-4593-4d2e-9b2c-892f696e0c43 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:42.730488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-af47d7a9-4593-4d2e-9b2c-892f696e0c43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:42.731573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302e18d-cac8-4704-95cb-0a8bd491e1a2 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:42.735479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c64c35-6811-43a5-9d3b-1e0de19e43e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:42.736156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c64c35-6811-43a5-9d3b-1e0de19e43e6 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.736525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c64c35-6811-43a5-9d3b-1e0de19e43e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.736901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c64c35-6811-43a5-9d3b-1e0de19e43e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.742876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47d7a9-4593-4d2e-9b2c-892f696e0c43 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:42.747225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cfdfc-e117-4834-85a6-6829ee1a27df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:42.747829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cfdfc-e117-4834-85a6-6829ee1a27df service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:42.748123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cfdfc-e117-4834-85a6-6829ee1a27df service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:42.748422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cfdfc-e117-4834-85a6-6829ee1a27df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:42.794249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:42.797955 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c64c35-6811-43a5-9d3b-1e0de19e43e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:42.799963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c64c35-6811-43a5-9d3b-1e0de19e43e6 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:42.807971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cfdfc-e117-4834-85a6-6829ee1a27df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:42.809448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cfdfc-e117-4834-85a6-6829ee1a27df service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:42.809973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 812/1639] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:42.818703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 828/1640] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:42 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:41:49.576857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c64c35-6811-43a5-9d3b-1e0de19e43e6 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:49.643613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c64c35-6811-43a5-9d3b-1e0de19e43e6 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:49.648465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:49.649112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:49.649971 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:49.650256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] PATH_INFO: `/v3/users/38bfae8a463a4a329a927a14615aeac7` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:49.652003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38bfae8a463a4a329a927a14615aeac7)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:49.654136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:49.753520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38bfae8a463a4a329a927a14615aeac7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:49.762910 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38bfae8a463a4a329a927a14615aeac7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:49.763688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38bfae8a463a4a329a927a14615aeac7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:49.764464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38bfae8a463a4a329a927a14615aeac7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:49.778043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38bfae8a463a4a329a927a14615aeac7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:49.802885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 813/1641] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:49 2022] DELETE /identity/v3/users/38bfae8a463a4a329a927a14615aeac7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:49.808373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d4122-56fa-4b60-8f1d-47a3bb8f73bf None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:49.810554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d4122-56fa-4b60-8f1d-47a3bb8f73bf None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:49.811023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d4122-56fa-4b60-8f1d-47a3bb8f73bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:49.846210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-485d4122-56fa-4b60-8f1d-47a3bb8f73bf None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:49.847597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-485d4122-56fa-4b60-8f1d-47a3bb8f73bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:49.931462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 829/1642] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:49.943081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:49.948673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:50.020638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82655d1e-3af7-452a-9047-d73e16807664 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:50.025192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443964d9-be8c-42f2-9842-294ce5a78141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:50.025956 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443964d9-be8c-42f2-9842-294ce5a78141 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:50.026890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443964d9-be8c-42f2-9842-294ce5a78141 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:50.027157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443964d9-be8c-42f2-9842-294ce5a78141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:50.040310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443964d9-be8c-42f2-9842-294ce5a78141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:50.043114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443964d9-be8c-42f2-9842-294ce5a78141 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:50.062614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 814/1643] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:50.398902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d4122-56fa-4b60-8f1d-47a3bb8f73bf None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:50.400440 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-485d4122-56fa-4b60-8f1d-47a3bb8f73bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:50.412685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d4122-56fa-4b60-8f1d-47a3bb8f73bf None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:50.418328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:50.419241 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:50.419812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:50.420099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] PATH_INFO: `/v3/projects/6328839e04cb4b1fae432022e904e98c` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:50.421297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6328839e04cb4b1fae432022e904e98c)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:50.423872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:50.488620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6328839e04cb4b1fae432022e904e98c', 'request_id': 'req-1284d056-008d-43ca-8785-ab1f881a54e1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.494425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6328839e04cb4b1fae432022e904e98c', 'request_id': 'req-1284d056-008d-43ca-8785-ab1f881a54e1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.502445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6328839e04cb4b1fae432022e904e98c', 'request_id': 'req-1284d056-008d-43ca-8785-ab1f881a54e1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.514857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] The token cache is being invalidate because project 6328839e04cb4b1fae432022e904e98c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:50.515151 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.515801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.517327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 830/1644] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:50 2022] DELETE /identity/v3/projects/6328839e04cb4b1fae432022e904e98c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:50.521443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443964d9-be8c-42f2-9842-294ce5a78141 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:50.531508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443964d9-be8c-42f2-9842-294ce5a78141 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:50.544070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:50.596924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:50.597652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:50.598005 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:50.598471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] PATH_INFO: `/v3/users/6d05bb2586e14ef186d56ff7e4899bcb` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:50.599682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d05bb2586e14ef186d56ff7e4899bcb)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:50.601756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:50.649782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d05bb2586e14ef186d56ff7e4899bcb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.656793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d05bb2586e14ef186d56ff7e4899bcb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.657484 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d05bb2586e14ef186d56ff7e4899bcb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.658086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d05bb2586e14ef186d56ff7e4899bcb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.665767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-16e31f6c-41ff-4fd9-8233-4a9ffedc9aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d05bb2586e14ef186d56ff7e4899bcb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:50.683789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 815/1645] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:50 2022] DELETE /identity/v3/users/6d05bb2586e14ef186d56ff7e4899bcb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:50.990702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:50.992831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:51.006814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284d056-008d-43ca-8785-ab1f881a54e1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:51.017491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:51.076433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:51.077067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.077342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.077609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] PATH_INFO: `/v3/projects/b249398b7a04483bb8ce1e5544f3dddf` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.078716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b249398b7a04483bb8ce1e5544f3dddf)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:51.080635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:51.127614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b249398b7a04483bb8ce1e5544f3dddf', 'request_id': 'req-b453578b-c214-4073-a221-edf118c5daa0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:51.133102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b249398b7a04483bb8ce1e5544f3dddf', 'request_id': 'req-b453578b-c214-4073-a221-edf118c5daa0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:51.142066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b249398b7a04483bb8ce1e5544f3dddf', 'request_id': 'req-b453578b-c214-4073-a221-edf118c5daa0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:51.153953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] The token cache is being invalidate because project b249398b7a04483bb8ce1e5544f3dddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:51.154230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:51.154882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b453578b-c214-4073-a221-edf118c5daa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:51.160701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 831/1646] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:50 2022] DELETE /identity/v3/projects/b249398b7a04483bb8ce1e5544f3dddf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:51.162378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69de7fb-68ae-40f4-b0ee-80db09b950c1 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.162665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69de7fb-68ae-40f4-b0ee-80db09b950c1 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.163094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69de7fb-68ae-40f4-b0ee-80db09b950c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.183748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-e69de7fb-68ae-40f4-b0ee-80db09b950c1 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:51.185101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e69de7fb-68ae-40f4-b0ee-80db09b950c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:51.255144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 816/1647] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:51.263447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729316f-4eed-4187-9ed6-268b71d518f4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.263788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729316f-4eed-4187-9ed6-268b71d518f4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.264094 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729316f-4eed-4187-9ed6-268b71d518f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.284244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-a729316f-4eed-4187-9ed6-268b71d518f4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:51.285482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a729316f-4eed-4187-9ed6-268b71d518f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:51.293332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 832/1648] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:51.298149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69de7fb-68ae-40f4-b0ee-80db09b950c1 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:51.307599 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69de7fb-68ae-40f4-b0ee-80db09b950c1 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:51.312321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf9f8-2373-4384-a8c4-8942efd173d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:51.312964 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf9f8-2373-4384-a8c4-8942efd173d3 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.313244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf9f8-2373-4384-a8c4-8942efd173d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.313518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf9f8-2373-4384-a8c4-8942efd173d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.314168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bf9f8-2373-4384-a8c4-8942efd173d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:51.314474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bf9f8-2373-4384-a8c4-8942efd173d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:51.316080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bf9f8-2373-4384-a8c4-8942efd173d3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:51.328452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 817/1649] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:51.332686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729316f-4eed-4187-9ed6-268b71d518f4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:51.342898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729316f-4eed-4187-9ed6-268b71d518f4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:51.347944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5130ae-5b85-4962-903d-9b376e9f88b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:51.348681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5130ae-5b85-4962-903d-9b376e9f88b1 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.349020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5130ae-5b85-4962-903d-9b376e9f88b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.349250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5130ae-5b85-4962-903d-9b376e9f88b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.350023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5130ae-5b85-4962-903d-9b376e9f88b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:51.352015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5130ae-5b85-4962-903d-9b376e9f88b1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:51.397894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 833/1650] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:51 2022] POST /identity/v3/projects => generated 399 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:51.398560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf9f8-2373-4384-a8c4-8942efd173d3 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:51.415950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf9f8-2373-4384-a8c4-8942efd173d3 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:51.441809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:51.484075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e8b3f-bb89-44e5-b190-14d56945b99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:51.484776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e8b3f-bb89-44e5-b190-14d56945b99c admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.485058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e8b3f-bb89-44e5-b190-14d56945b99c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.485583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e8b3f-bb89-44e5-b190-14d56945b99c admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.486218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e8b3f-bb89-44e5-b190-14d56945b99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:51.488743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e8b3f-bb89-44e5-b190-14d56945b99c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:51.514265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 818/1651] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:51 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:51.518089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5130ae-5b85-4962-903d-9b376e9f88b1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:51.529556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5130ae-5b85-4962-903d-9b376e9f88b1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:51.534291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6c97b-03e5-4889-aff4-a89827e6a228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:51.534981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6c97b-03e5-4889-aff4-a89827e6a228 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.535260 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6c97b-03e5-4889-aff4-a89827e6a228 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.535530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6c97b-03e5-4889-aff4-a89827e6a228 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.536175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6c97b-03e5-4889-aff4-a89827e6a228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:51.537812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6c97b-03e5-4889-aff4-a89827e6a228 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:51.551002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 834/1652] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:51.553571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e8b3f-bb89-44e5-b190-14d56945b99c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:51.565347 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e8b3f-bb89-44e5-b190-14d56945b99c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:51.570216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba37736-fd06-431d-bf6f-bbac7d7670cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:51.570839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba37736-fd06-431d-bf6f-bbac7d7670cd admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.571121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba37736-fd06-431d-bf6f-bbac7d7670cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.571398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba37736-fd06-431d-bf6f-bbac7d7670cd admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.572054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba37736-fd06-431d-bf6f-bbac7d7670cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:51.574470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba37736-fd06-431d-bf6f-bbac7d7670cd admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:51.586141 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 819/1653] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:51.590425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6c97b-03e5-4889-aff4-a89827e6a228 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:51.601302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6c97b-03e5-4889-aff4-a89827e6a228 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:51.606785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4a68b-8893-40f8-965b-543aaa1a9364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:51.607542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4a68b-8893-40f8-965b-543aaa1a9364 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.607894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4a68b-8893-40f8-965b-543aaa1a9364 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.608240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4a68b-8893-40f8-965b-543aaa1a9364 admin admin] PATH_INFO: `/v3/projects/4dd2b494d34a482ba8bb2baa9c8a0791/users/54ab581a7bb143e1b1da168af81f1e1f/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.626299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4a68b-8893-40f8-965b-543aaa1a9364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dd2b494d34a482ba8bb2baa9c8a0791, user_id=54ab581a7bb143e1b1da168af81f1e1f, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:51.629133 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4a68b-8893-40f8-965b-543aaa1a9364 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:51.646656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 835/1654] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:51 2022] PUT /identity/v3/projects/4dd2b494d34a482ba8bb2baa9c8a0791/users/54ab581a7bb143e1b1da168af81f1e1f/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:51.657229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a9435-077f-474a-89a5-763bc38e2d48 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.657688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a9435-077f-474a-89a5-763bc38e2d48 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.657760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a9435-077f-474a-89a5-763bc38e2d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.687515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-8d8a9435-077f-474a-89a5-763bc38e2d48 None None] MFA Rules not processed for user `54ab581a7bb143e1b1da168af81f1e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:51.688763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8d8a9435-077f-474a-89a5-763bc38e2d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:51.759122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 820/1655] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:51.764158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4a68b-8893-40f8-965b-543aaa1a9364 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:51.809212 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:51.837841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4a68b-8893-40f8-965b-543aaa1a9364 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:51.844286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a50d66-4432-4cd0-9a9f-7d0a88875f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:51.844975 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a50d66-4432-4cd0-9a9f-7d0a88875f82 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.845244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a50d66-4432-4cd0-9a9f-7d0a88875f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.845524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a50d66-4432-4cd0-9a9f-7d0a88875f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.846346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a50d66-4432-4cd0-9a9f-7d0a88875f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:51.848293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a50d66-4432-4cd0-9a9f-7d0a88875f82 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:51.889335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 836/1656] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:51 2022] POST /identity/v3/projects => generated 401 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:51.893421 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a9435-077f-474a-89a5-763bc38e2d48 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:51.905277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a9435-077f-474a-89a5-763bc38e2d48 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:51.974106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43e5ab-bdf2-41eb-b5db-322bf7f4bead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:51.974733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43e5ab-bdf2-41eb-b5db-322bf7f4bead admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:51.975014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43e5ab-bdf2-41eb-b5db-322bf7f4bead admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:51.975296 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43e5ab-bdf2-41eb-b5db-322bf7f4bead admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:51.976006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43e5ab-bdf2-41eb-b5db-322bf7f4bead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:51.978030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43e5ab-bdf2-41eb-b5db-322bf7f4bead admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:52.011414 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 821/1657] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:51 2022] POST /identity/v3/users => generated 307 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:52.014907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a50d66-4432-4cd0-9a9f-7d0a88875f82 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:52.028211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a50d66-4432-4cd0-9a9f-7d0a88875f82 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:52.047052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea965b6-cb27-4b23-a05d-a39e52d79d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:52.047742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea965b6-cb27-4b23-a05d-a39e52d79d3b admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:52.048035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea965b6-cb27-4b23-a05d-a39e52d79d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:52.048315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea965b6-cb27-4b23-a05d-a39e52d79d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:52.050265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea965b6-cb27-4b23-a05d-a39e52d79d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:52.054589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea965b6-cb27-4b23-a05d-a39e52d79d3b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:52.082088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 837/1658] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:52.086211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43e5ab-bdf2-41eb-b5db-322bf7f4bead admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:52.096819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43e5ab-bdf2-41eb-b5db-322bf7f4bead admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:52.101965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f629f53-8b04-4e9f-b1e7-b747697a81a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:52.102641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f629f53-8b04-4e9f-b1e7-b747697a81a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:52.102925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f629f53-8b04-4e9f-b1e7-b747697a81a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:52.103203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f629f53-8b04-4e9f-b1e7-b747697a81a0 admin admin] PATH_INFO: `/v3/projects/b48cccec29ec4b2b8c039f6343d628ea/users/05d2b70009e0433e8abacdac5b60b6df/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:52.122901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f629f53-8b04-4e9f-b1e7-b747697a81a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48cccec29ec4b2b8c039f6343d628ea, user_id=05d2b70009e0433e8abacdac5b60b6df, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:52.125143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f629f53-8b04-4e9f-b1e7-b747697a81a0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:52.128853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:52.147979 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 822/1659] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:52 2022] PUT /identity/v3/projects/b48cccec29ec4b2b8c039f6343d628ea/users/05d2b70009e0433e8abacdac5b60b6df/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:52.149426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea965b6-cb27-4b23-a05d-a39e52d79d3b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:52.162626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea965b6-cb27-4b23-a05d-a39e52d79d3b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:52.226949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1c3df-760f-434d-9cb8-e7572d5af485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:52.227562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1c3df-760f-434d-9cb8-e7572d5af485 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:52.227910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1c3df-760f-434d-9cb8-e7572d5af485 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:52.228113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1c3df-760f-434d-9cb8-e7572d5af485 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:52.228782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1c3df-760f-434d-9cb8-e7572d5af485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:52.231068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1c3df-760f-434d-9cb8-e7572d5af485 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:52.241818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 838/1660] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:52.246620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f629f53-8b04-4e9f-b1e7-b747697a81a0 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:52.260837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f629f53-8b04-4e9f-b1e7-b747697a81a0 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:52.264278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8875c4c-5e32-43e7-838d-f50cecd1e433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:52.264921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8875c4c-5e32-43e7-838d-f50cecd1e433 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:52.265200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8875c4c-5e32-43e7-838d-f50cecd1e433 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:52.265474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8875c4c-5e32-43e7-838d-f50cecd1e433 admin admin] PATH_INFO: `/v3/projects/b48cccec29ec4b2b8c039f6343d628ea/users/05d2b70009e0433e8abacdac5b60b6df/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:52.267694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8875c4c-5e32-43e7-838d-f50cecd1e433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48cccec29ec4b2b8c039f6343d628ea, user_id=05d2b70009e0433e8abacdac5b60b6df, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:52.270466 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8875c4c-5e32-43e7-838d-f50cecd1e433 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:52.281859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 823/1661] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:52 2022] PUT /identity/v3/projects/b48cccec29ec4b2b8c039f6343d628ea/users/05d2b70009e0433e8abacdac5b60b6df/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:52.293249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d6ee5-a3ce-4bc5-b18c-11f74c2ab984 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:52.293616 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d6ee5-a3ce-4bc5-b18c-11f74c2ab984 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:52.293893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d6ee5-a3ce-4bc5-b18c-11f74c2ab984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:52.328719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9c6d6ee5-a3ce-4bc5-b18c-11f74c2ab984 None None] MFA Rules not processed for user `05d2b70009e0433e8abacdac5b60b6df`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:52.329986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9c6d6ee5-a3ce-4bc5-b18c-11f74c2ab984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:52.410754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 839/1662] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:52 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:52.421831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8875c4c-5e32-43e7-838d-f50cecd1e433 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:52.423718 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d8875c4c-5e32-43e7-838d-f50cecd1e433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:52.497633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8875c4c-5e32-43e7-838d-f50cecd1e433 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:52.502493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83c0b-b8d0-4ad3-bb1f-c28daf453ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:52.505123 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83c0b-b8d0-4ad3-bb1f-c28daf453ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:52.505541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83c0b-b8d0-4ad3-bb1f-c28daf453ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:52.505827 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83c0b-b8d0-4ad3-bb1f-c28daf453ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:52.518296 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec83c0b-b8d0-4ad3-bb1f-c28daf453ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:52.520531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec83c0b-b8d0-4ad3-bb1f-c28daf453ff1 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:52.533530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 824/1663] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:52 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:52.615033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d6ee5-a3ce-4bc5-b18c-11f74c2ab984 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:52.625311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d6ee5-a3ce-4bc5-b18c-11f74c2ab984 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:52.629768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259b74f-0df2-447b-8f19-7d73c9570d67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:52.630449 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259b74f-0df2-447b-8f19-7d73c9570d67 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:52.630733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259b74f-0df2-447b-8f19-7d73c9570d67 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:52.631013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259b74f-0df2-447b-8f19-7d73c9570d67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:52.668277 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:52.696758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259b74f-0df2-447b-8f19-7d73c9570d67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:52.699027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259b74f-0df2-447b-8f19-7d73c9570d67 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:52.709108 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 840/1664] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:52 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:54.628355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83c0b-b8d0-4ad3-bb1f-c28daf453ff1 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:54.629819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bec83c0b-b8d0-4ad3-bb1f-c28daf453ff1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:54.708973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83c0b-b8d0-4ad3-bb1f-c28daf453ff1 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:54.715708 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:54.716505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:54.716859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:54.717142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] PATH_INFO: `/v3/users/243452abe98040de9ee11ddf513c87c3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:54.718295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] RBAC: Authorizing `identity:delete_user(user_id=243452abe98040de9ee11ddf513c87c3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:54.720268 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:54.783873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '243452abe98040de9ee11ddf513c87c3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:54.802185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '243452abe98040de9ee11ddf513c87c3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:54.802997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '243452abe98040de9ee11ddf513c87c3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:54.803618 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '243452abe98040de9ee11ddf513c87c3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:54.815698 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '243452abe98040de9ee11ddf513c87c3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:54.874455 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 825/1665] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:54 2022] DELETE /identity/v3/users/243452abe98040de9ee11ddf513c87c3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:54.879711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174f37-a228-483a-ae9b-5e537cfcabd0 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:54.880306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174f37-a228-483a-ae9b-5e537cfcabd0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:54.880306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174f37-a228-483a-ae9b-5e537cfcabd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:54.914506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ed174f37-a228-483a-ae9b-5e537cfcabd0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:54.916533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ed174f37-a228-483a-ae9b-5e537cfcabd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:55.030439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 841/1666] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:55.049567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:55.051396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:55.061860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:55.182859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c827ea-9e13-454f-b726-296b83a3b52b admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:55.187103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8952e8e-61f1-4d21-848b-42ab989f6fee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:55.187775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8952e8e-61f1-4d21-848b-42ab989f6fee service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:55.188061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8952e8e-61f1-4d21-848b-42ab989f6fee service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:55.188337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8952e8e-61f1-4d21-848b-42ab989f6fee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:55.200100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8952e8e-61f1-4d21-848b-42ab989f6fee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:55.201677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8952e8e-61f1-4d21-848b-42ab989f6fee service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:55.214915 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 826/1667] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:56.360763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed174f37-a228-483a-ae9b-5e537cfcabd0 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:56.362304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ed174f37-a228-483a-ae9b-5e537cfcabd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:56.380218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed174f37-a228-483a-ae9b-5e537cfcabd0 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:56.398764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:56.399854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:56.400373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:56.403112 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] PATH_INFO: `/v3/projects/5f2d05ef79b74db9b589af9ca0a63f1a` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:56.405002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2d05ef79b74db9b589af9ca0a63f1a)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:56.407130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:56.490670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2d05ef79b74db9b589af9ca0a63f1a', 'request_id': 'req-a1b43263-8641-4130-85d6-9e9ec846fc6b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.497712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2d05ef79b74db9b589af9ca0a63f1a', 'request_id': 'req-a1b43263-8641-4130-85d6-9e9ec846fc6b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.507357 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2d05ef79b74db9b589af9ca0a63f1a', 'request_id': 'req-a1b43263-8641-4130-85d6-9e9ec846fc6b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.523017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] The token cache is being invalidate because project 5f2d05ef79b74db9b589af9ca0a63f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:56.523540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.524971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.527838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 842/1668] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:56 2022] DELETE /identity/v3/projects/5f2d05ef79b74db9b589af9ca0a63f1a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:56.531777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8952e8e-61f1-4d21-848b-42ab989f6fee service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:56.541801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8952e8e-61f1-4d21-848b-42ab989f6fee service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:56.602508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:56.603157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:56.603436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:56.603710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] PATH_INFO: `/v3/users/af937f8306cc4874b6c7dbde68b0ccb2` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:56.605000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=af937f8306cc4874b6c7dbde68b0ccb2)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:56.607002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:56.661734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af937f8306cc4874b6c7dbde68b0ccb2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.671604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af937f8306cc4874b6c7dbde68b0ccb2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.672374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af937f8306cc4874b6c7dbde68b0ccb2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.673262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af937f8306cc4874b6c7dbde68b0ccb2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.682023 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3f431cf1-584e-473f-afa1-71e492517bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af937f8306cc4874b6c7dbde68b0ccb2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:56.704968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 827/1669] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:56 2022] DELETE /identity/v3/users/af937f8306cc4874b6c7dbde68b0ccb2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:57.087786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:57.090783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:57.112330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b43263-8641-4130-85d6-9e9ec846fc6b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:57.180369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:57.190282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:57.191143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.192386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.192588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] PATH_INFO: `/v3/projects/75081191b01b4456bdab2e43ff64f547` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.193836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] RBAC: Authorizing `identity:delete_project(project_id=75081191b01b4456bdab2e43ff64f547)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:57.195962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:57.266939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75081191b01b4456bdab2e43ff64f547', 'request_id': 'req-fd6ea37b-c9b5-4a33-91ac-b825e701130e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:57.280889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75081191b01b4456bdab2e43ff64f547', 'request_id': 'req-fd6ea37b-c9b5-4a33-91ac-b825e701130e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:57.290999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75081191b01b4456bdab2e43ff64f547', 'request_id': 'req-fd6ea37b-c9b5-4a33-91ac-b825e701130e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:57.304619 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] The token cache is being invalidate because project 75081191b01b4456bdab2e43ff64f547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:41:57.304922 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:57.305592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd6ea37b-c9b5-4a33-91ac-b825e701130e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:57.307723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 843/1670] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:41:57 2022] DELETE /identity/v3/projects/75081191b01b4456bdab2e43ff64f547 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:57.312527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55511efe-6ac8-4a03-bee5-48131edef602 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.312807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55511efe-6ac8-4a03-bee5-48131edef602 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.313090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55511efe-6ac8-4a03-bee5-48131edef602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.341593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-55511efe-6ac8-4a03-bee5-48131edef602 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:57.342865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-55511efe-6ac8-4a03-bee5-48131edef602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:57.379100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:57.437717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 828/1671] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:57.445506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e9c3-0a34-4110-b3c0-d2d2a456caa5 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.446035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e9c3-0a34-4110-b3c0-d2d2a456caa5 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.446035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e9c3-0a34-4110-b3c0-d2d2a456caa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.467011 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-8144e9c3-0a34-4110-b3c0-d2d2a456caa5 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:57.468663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8144e9c3-0a34-4110-b3c0-d2d2a456caa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:57.477249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 844/1672] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:57.482284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55511efe-6ac8-4a03-bee5-48131edef602 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:57.492931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55511efe-6ac8-4a03-bee5-48131edef602 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:57.500334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed195021-f0ba-4c61-aafa-87a694625cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:57.501003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed195021-f0ba-4c61-aafa-87a694625cca admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.501300 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed195021-f0ba-4c61-aafa-87a694625cca admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.501558 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed195021-f0ba-4c61-aafa-87a694625cca admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.502216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed195021-f0ba-4c61-aafa-87a694625cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:41:57.502672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed195021-f0ba-4c61-aafa-87a694625cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:57.504324 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed195021-f0ba-4c61-aafa-87a694625cca admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:57.520533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 829/1673] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:41:57.524858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144e9c3-0a34-4110-b3c0-d2d2a456caa5 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:57.534682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144e9c3-0a34-4110-b3c0-d2d2a456caa5 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:57.539621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c62e5-7600-4903-9da6-c107cd528128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:57.540381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c62e5-7600-4903-9da6-c107cd528128 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.540584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c62e5-7600-4903-9da6-c107cd528128 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.540823 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c62e5-7600-4903-9da6-c107cd528128 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.541490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c62e5-7600-4903-9da6-c107cd528128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:57.543531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c62e5-7600-4903-9da6-c107cd528128 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:57.575660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 845/1674] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:57 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:57.579640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed195021-f0ba-4c61-aafa-87a694625cca admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:57.592050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed195021-f0ba-4c61-aafa-87a694625cca admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:57.654971 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ecc34-f6ff-46ca-a27b-deae5d1ea37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:57.655712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ecc34-f6ff-46ca-a27b-deae5d1ea37f admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.656019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ecc34-f6ff-46ca-a27b-deae5d1ea37f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.656295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ecc34-f6ff-46ca-a27b-deae5d1ea37f admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.656965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ecc34-f6ff-46ca-a27b-deae5d1ea37f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:57.658885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ecc34-f6ff-46ca-a27b-deae5d1ea37f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:57.686572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 830/1675] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:57 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:57.687976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c62e5-7600-4903-9da6-c107cd528128 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:57.698537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c62e5-7600-4903-9da6-c107cd528128 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:57.703125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89fecb-26ac-4df4-97e3-503ba6dbb5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:57.703732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89fecb-26ac-4df4-97e3-503ba6dbb5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.704013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89fecb-26ac-4df4-97e3-503ba6dbb5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.704284 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89fecb-26ac-4df4-97e3-503ba6dbb5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.704958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b89fecb-26ac-4df4-97e3-503ba6dbb5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:57.706593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b89fecb-26ac-4df4-97e3-503ba6dbb5e2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:57.717961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 846/1676] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:57.721622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ecc34-f6ff-46ca-a27b-deae5d1ea37f admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:57.734567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ecc34-f6ff-46ca-a27b-deae5d1ea37f admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:57.739789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876dd4e-5ad5-4250-a547-243246933f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:57.740449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876dd4e-5ad5-4250-a547-243246933f58 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.740735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876dd4e-5ad5-4250-a547-243246933f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.741396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876dd4e-5ad5-4250-a547-243246933f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.741943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876dd4e-5ad5-4250-a547-243246933f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:57.743560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876dd4e-5ad5-4250-a547-243246933f58 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:57.760755 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 831/1677] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:57.761323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89fecb-26ac-4df4-97e3-503ba6dbb5e2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:57.771601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89fecb-26ac-4df4-97e3-503ba6dbb5e2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:57.776974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0c148-bd87-47ed-acb2-5273d0e8d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:57.777643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0c148-bd87-47ed-acb2-5273d0e8d57d admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.777928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0c148-bd87-47ed-acb2-5273d0e8d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.778204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0c148-bd87-47ed-acb2-5273d0e8d57d admin admin] PATH_INFO: `/v3/projects/8db062fc1ab147a5a513f6f8d776be21/users/c093bc7e38bf494495e917322b44d0a4/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.794550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0c148-bd87-47ed-acb2-5273d0e8d57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db062fc1ab147a5a513f6f8d776be21, user_id=c093bc7e38bf494495e917322b44d0a4, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:57.797222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0c148-bd87-47ed-acb2-5273d0e8d57d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:57.818410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 847/1678] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:57 2022] PUT /identity/v3/projects/8db062fc1ab147a5a513f6f8d776be21/users/c093bc7e38bf494495e917322b44d0a4/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:57.819882 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ff6ea-cc52-493e-a297-f2ede00770d3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.820176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ff6ea-cc52-493e-a297-f2ede00770d3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.820489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ff6ea-cc52-493e-a297-f2ede00770d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.841547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-268ff6ea-cc52-493e-a297-f2ede00770d3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:57.842775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-268ff6ea-cc52-493e-a297-f2ede00770d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:57.901935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 832/1679] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:41:57.911514 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0c148-bd87-47ed-acb2-5273d0e8d57d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:57.913319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b5c0c148-bd87-47ed-acb2-5273d0e8d57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:57.971041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0c148-bd87-47ed-acb2-5273d0e8d57d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:57.974745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d90fd0-f17a-4909-a814-2c892c7a406e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:57.975360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d90fd0-f17a-4909-a814-2c892c7a406e service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:57.975646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d90fd0-f17a-4909-a814-2c892c7a406e service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:57.975924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d90fd0-f17a-4909-a814-2c892c7a406e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:57.985560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d90fd0-f17a-4909-a814-2c892c7a406e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:57.987054 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d90fd0-f17a-4909-a814-2c892c7a406e service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:57.996994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 848/1680] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:41:58.980129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da9a60-37d3-4af4-b4be-9324a8323611 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:58.980598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da9a60-37d3-4af4-b4be-9324a8323611 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:58.980755 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da9a60-37d3-4af4-b4be-9324a8323611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.020096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-91da9a60-37d3-4af4-b4be-9324a8323611 None None] MFA Rules not processed for user `c093bc7e38bf494495e917322b44d0a4`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:59.021244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-91da9a60-37d3-4af4-b4be-9324a8323611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:59.045317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:59.098748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 833/1681] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:41:59.100502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d90fd0-f17a-4909-a814-2c892c7a406e service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:59.111921 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d90fd0-f17a-4909-a814-2c892c7a406e service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:59.117739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa629ae-7ece-4611-9e6a-5271dec0ef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:59.118575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa629ae-7ece-4611-9e6a-5271dec0ef16 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:59.119137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa629ae-7ece-4611-9e6a-5271dec0ef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:59.120281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa629ae-7ece-4611-9e6a-5271dec0ef16 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.121331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa629ae-7ece-4611-9e6a-5271dec0ef16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:59.124259 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa629ae-7ece-4611-9e6a-5271dec0ef16 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:59.156310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 849/1682] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:41:59 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:41:59.157597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da9a60-37d3-4af4-b4be-9324a8323611 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:59.168627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da9a60-37d3-4af4-b4be-9324a8323611 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:59.227676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723ef71-b4f7-49d0-a098-910dd3bdbe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:59.228242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ef71-b4f7-49d0-a098-910dd3bdbe0a admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:59.228558 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ef71-b4f7-49d0-a098-910dd3bdbe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:59.228839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ef71-b4f7-49d0-a098-910dd3bdbe0a admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.229773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723ef71-b4f7-49d0-a098-910dd3bdbe0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:59.231723 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723ef71-b4f7-49d0-a098-910dd3bdbe0a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:59.256208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 834/1683] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:41:59 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:41:59.261770 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa629ae-7ece-4611-9e6a-5271dec0ef16 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:59.268129 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:59.272434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa629ae-7ece-4611-9e6a-5271dec0ef16 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:59.279090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad51091-de98-4c48-afc3-bfc952c4d1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:59.279883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad51091-de98-4c48-afc3-bfc952c4d1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:59.280567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad51091-de98-4c48-afc3-bfc952c4d1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:59.281100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad51091-de98-4c48-afc3-bfc952c4d1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.281705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad51091-de98-4c48-afc3-bfc952c4d1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:59.283418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad51091-de98-4c48-afc3-bfc952c4d1c2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:59.297645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 850/1684] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:41:59.301900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723ef71-b4f7-49d0-a098-910dd3bdbe0a admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:59.314359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723ef71-b4f7-49d0-a098-910dd3bdbe0a admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:59.322572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e420b47-51cb-46fe-be9a-1ab15e7944af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:59.323979 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e420b47-51cb-46fe-be9a-1ab15e7944af admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:59.324436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e420b47-51cb-46fe-be9a-1ab15e7944af admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:59.324711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e420b47-51cb-46fe-be9a-1ab15e7944af admin admin] PATH_INFO: `/v3/projects/c6620e37a86642dead8bffe716fc6bac/users/6e2ec1218d724b4caf1a98e589efbfa0/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.358297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e420b47-51cb-46fe-be9a-1ab15e7944af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6620e37a86642dead8bffe716fc6bac, user_id=6e2ec1218d724b4caf1a98e589efbfa0, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:59.361367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e420b47-51cb-46fe-be9a-1ab15e7944af admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:59.377110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 835/1685] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:59 2022] PUT /identity/v3/projects/c6620e37a86642dead8bffe716fc6bac/users/6e2ec1218d724b4caf1a98e589efbfa0/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:59.381805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad51091-de98-4c48-afc3-bfc952c4d1c2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:59.395075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad51091-de98-4c48-afc3-bfc952c4d1c2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:59.477766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e420b47-51cb-46fe-be9a-1ab15e7944af admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:59.479518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3e420b47-51cb-46fe-be9a-1ab15e7944af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:59.490053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9002d-9ac8-4f2b-bd91-cdb7c3946c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:59.492182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9002d-9ac8-4f2b-bd91-cdb7c3946c25 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:59.493123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9002d-9ac8-4f2b-bd91-cdb7c3946c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:59.493705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9002d-9ac8-4f2b-bd91-cdb7c3946c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.495564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9002d-9ac8-4f2b-bd91-cdb7c3946c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:59.496277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e420b47-51cb-46fe-be9a-1ab15e7944af admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:59.502978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:59.503680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:59.503967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:59.504256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] PATH_INFO: `/v3/users/54ab581a7bb143e1b1da168af81f1e1f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.508519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ab581a7bb143e1b1da168af81f1e1f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:59.511434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:59.514685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9002d-9ac8-4f2b-bd91-cdb7c3946c25 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:59.534427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 851/1686] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:41:59.539080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9002d-9ac8-4f2b-bd91-cdb7c3946c25 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:59.583306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9002d-9ac8-4f2b-bd91-cdb7c3946c25 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:59.591183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1eda86-bcb5-491b-9696-dc63eaba1ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:59.592320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eda86-bcb5-491b-9696-dc63eaba1ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:59.593441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eda86-bcb5-491b-9696-dc63eaba1ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:59.594031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eda86-bcb5-491b-9696-dc63eaba1ab7 admin admin] PATH_INFO: `/v3/projects/c6620e37a86642dead8bffe716fc6bac/users/6e2ec1218d724b4caf1a98e589efbfa0/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.596477 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ab581a7bb143e1b1da168af81f1e1f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:59.597806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1eda86-bcb5-491b-9696-dc63eaba1ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6620e37a86642dead8bffe716fc6bac, user_id=6e2ec1218d724b4caf1a98e589efbfa0, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:59.600657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1eda86-bcb5-491b-9696-dc63eaba1ab7 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:59.607867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ab581a7bb143e1b1da168af81f1e1f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:59.608669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ab581a7bb143e1b1da168af81f1e1f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:59.609326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ab581a7bb143e1b1da168af81f1e1f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:59.617206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 852/1687] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:41:59 2022] PUT /identity/v3/projects/c6620e37a86642dead8bffe716fc6bac/users/6e2ec1218d724b4caf1a98e589efbfa0/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:41:59.622139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ab581a7bb143e1b1da168af81f1e1f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:41:59.641532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:41:59.647993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 836/1688] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:41:59 2022] DELETE /identity/v3/users/54ab581a7bb143e1b1da168af81f1e1f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:41:59.653766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cebc4-58ff-479a-8898-157c803b00b2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:59.654115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cebc4-58ff-479a-8898-157c803b00b2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:59.654646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cebc4-58ff-479a-8898-157c803b00b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.678893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-bb4cebc4-58ff-479a-8898-157c803b00b2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:41:59.680425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bb4cebc4-58ff-479a-8898-157c803b00b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:59.763434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 853/1689] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:41:59.770506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:41:59.772323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:41:59.853874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c909c4-8bb5-4176-8e63-73c02ba10af7 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:41:59.860384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d148fb-ab3b-468b-9d43-5b3e5fd0d391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:41:59.861136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d148fb-ab3b-468b-9d43-5b3e5fd0d391 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:41:59.861927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d148fb-ab3b-468b-9d43-5b3e5fd0d391 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:41:59.861927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d148fb-ab3b-468b-9d43-5b3e5fd0d391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:41:59.878070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d148fb-ab3b-468b-9d43-5b3e5fd0d391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:41:59.880016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d148fb-ab3b-468b-9d43-5b3e5fd0d391 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:41:59.893725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 837/1690] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:41:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:00.286158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cebc4-58ff-479a-8898-157c803b00b2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:00.287622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bb4cebc4-58ff-479a-8898-157c803b00b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:00.301922 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cebc4-58ff-479a-8898-157c803b00b2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:00.306993 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:00.307741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:00.308221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:00.308768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] PATH_INFO: `/v3/projects/4dd2b494d34a482ba8bb2baa9c8a0791` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:00.310186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dd2b494d34a482ba8bb2baa9c8a0791)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:00.312237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:00.383298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dd2b494d34a482ba8bb2baa9c8a0791', 'request_id': 'req-4518e64e-5f5b-4239-a229-d0fe92088613'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.389714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dd2b494d34a482ba8bb2baa9c8a0791', 'request_id': 'req-4518e64e-5f5b-4239-a229-d0fe92088613'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.398893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dd2b494d34a482ba8bb2baa9c8a0791', 'request_id': 'req-4518e64e-5f5b-4239-a229-d0fe92088613'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.408407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:00.412624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] The token cache is being invalidate because project 4dd2b494d34a482ba8bb2baa9c8a0791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:00.414342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.415858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.420432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 854/1691] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:00 2022] DELETE /identity/v3/projects/4dd2b494d34a482ba8bb2baa9c8a0791 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:00.430195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770cfbf-d35f-4f3a-9e40-77bee4633986 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:00.430996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770cfbf-d35f-4f3a-9e40-77bee4633986 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:00.431460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770cfbf-d35f-4f3a-9e40-77bee4633986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:00.437516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:00.439228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:00.468271 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-4770cfbf-d35f-4f3a-9e40-77bee4633986 None None] MFA Rules not processed for user `6e2ec1218d724b4caf1a98e589efbfa0`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:00.471059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4770cfbf-d35f-4f3a-9e40-77bee4633986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:00.515569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518e64e-5f5b-4239-a229-d0fe92088613 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:00.520866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:00.521568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:00.521777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:00.522052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] PATH_INFO: `/v3/users/05d2b70009e0433e8abacdac5b60b6df` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:00.523191 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d2b70009e0433e8abacdac5b60b6df)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:00.525174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:00.571465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 838/1692] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:00.580050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770cfbf-d35f-4f3a-9e40-77bee4633986 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:00.582104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4770cfbf-d35f-4f3a-9e40-77bee4633986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:00.602108 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d2b70009e0433e8abacdac5b60b6df'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.614243 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d2b70009e0433e8abacdac5b60b6df'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.615501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d2b70009e0433e8abacdac5b60b6df'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.616327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d2b70009e0433e8abacdac5b60b6df'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.627435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d2b70009e0433e8abacdac5b60b6df'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:00.666003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770cfbf-d35f-4f3a-9e40-77bee4633986 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:00.670141 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4d54b-c2ab-4bf9-b169-305649b38b49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:00.670748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4d54b-c2ab-4bf9-b169-305649b38b49 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:00.671050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4d54b-c2ab-4bf9-b169-305649b38b49 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:00.671338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4d54b-c2ab-4bf9-b169-305649b38b49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:00.673114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-76d4d54b-c2ab-4bf9-b169-305649b38b49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:00.673801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 855/1693] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:00 2022] DELETE /identity/v3/users/05d2b70009e0433e8abacdac5b60b6df => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:00.722108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:00.754244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4d54b-c2ab-4bf9-b169-305649b38b49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:00.755532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4d54b-c2ab-4bf9-b169-305649b38b49 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:00.766862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 839/1694] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:42:00.992010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:00.993592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:01.091401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7045e8-ee57-44b1-bb2b-958129fdd411 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.097292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.098030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.098383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.098732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] PATH_INFO: `/v3/projects/b48cccec29ec4b2b8c039f6343d628ea` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.099960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48cccec29ec4b2b8c039f6343d628ea)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.102076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.136557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:01.155438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48cccec29ec4b2b8c039f6343d628ea', 'request_id': 'req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:01.161593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48cccec29ec4b2b8c039f6343d628ea', 'request_id': 'req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:01.170308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48cccec29ec4b2b8c039f6343d628ea', 'request_id': 'req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:01.181755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] The token cache is being invalidate because project b48cccec29ec4b2b8c039f6343d628ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:01.182147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:01.182888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-fd1af1b9-b40e-404b-a471-e5fe991a1e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:01.184557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 856/1695] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:00 2022] DELETE /identity/v3/projects/b48cccec29ec4b2b8c039f6343d628ea => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:01.189878 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8699ad7-bf5e-4204-b371-8e9f13e9be93 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.190161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8699ad7-bf5e-4204-b371-8e9f13e9be93 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.190458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8699ad7-bf5e-4204-b371-8e9f13e9be93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.210837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-a8699ad7-bf5e-4204-b371-8e9f13e9be93 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:01.212015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a8699ad7-bf5e-4204-b371-8e9f13e9be93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:01.271533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 840/1696] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:01.280297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a71a1-ebd2-41c6-bb13-c22ff30423b4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.280630 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a71a1-ebd2-41c6-bb13-c22ff30423b4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.280900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a71a1-ebd2-41c6-bb13-c22ff30423b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.300406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9e4a71a1-ebd2-41c6-bb13-c22ff30423b4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:01.301613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9e4a71a1-ebd2-41c6-bb13-c22ff30423b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:01.310077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 857/1697] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:01.314346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8699ad7-bf5e-4204-b371-8e9f13e9be93 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.324393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8699ad7-bf5e-4204-b371-8e9f13e9be93 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.328946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0853505-8e16-4c59-840b-a5e33096b288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.329578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0853505-8e16-4c59-840b-a5e33096b288 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.329861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0853505-8e16-4c59-840b-a5e33096b288 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.330140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0853505-8e16-4c59-840b-a5e33096b288 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.330786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0853505-8e16-4c59-840b-a5e33096b288 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:42:01.331093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0853505-8e16-4c59-840b-a5e33096b288 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.332781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0853505-8e16-4c59-840b-a5e33096b288 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.347962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 841/1698] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:42:01.349156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a71a1-ebd2-41c6-bb13-c22ff30423b4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.359164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a71a1-ebd2-41c6-bb13-c22ff30423b4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.364166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fbf62-435d-4aac-933d-41826e4f5654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.364800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fbf62-435d-4aac-933d-41826e4f5654 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.365087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fbf62-435d-4aac-933d-41826e4f5654 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.365359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fbf62-435d-4aac-933d-41826e4f5654 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.366003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fbf62-435d-4aac-933d-41826e4f5654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.367937 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fbf62-435d-4aac-933d-41826e4f5654 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.395635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 858/1699] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:01 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:01.399850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0853505-8e16-4c59-840b-a5e33096b288 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.410474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0853505-8e16-4c59-840b-a5e33096b288 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.466890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b881b-ad9f-4c21-8338-287073511dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.467543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b881b-ad9f-4c21-8338-287073511dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.467760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b881b-ad9f-4c21-8338-287073511dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.468032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b881b-ad9f-4c21-8338-287073511dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.468694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b881b-ad9f-4c21-8338-287073511dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.470643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b881b-ad9f-4c21-8338-287073511dc4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.492185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 842/1700] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:01 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:01.496529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fbf62-435d-4aac-933d-41826e4f5654 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.507139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fbf62-435d-4aac-933d-41826e4f5654 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.512226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524192-7c80-478d-ba6f-d656485a9fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.512880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524192-7c80-478d-ba6f-d656485a9fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.513165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524192-7c80-478d-ba6f-d656485a9fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.513441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524192-7c80-478d-ba6f-d656485a9fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.514097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524192-7c80-478d-ba6f-d656485a9fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.515732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524192-7c80-478d-ba6f-d656485a9fa1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.526746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 859/1701] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:01.530744 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b881b-ad9f-4c21-8338-287073511dc4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.541163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b881b-ad9f-4c21-8338-287073511dc4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.549873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977e873-3424-4cdc-aeb6-3487bb375d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.550482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977e873-3424-4cdc-aeb6-3487bb375d58 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.550765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977e873-3424-4cdc-aeb6-3487bb375d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.551039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977e873-3424-4cdc-aeb6-3487bb375d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.551680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977e873-3424-4cdc-aeb6-3487bb375d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.553313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977e873-3424-4cdc-aeb6-3487bb375d58 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.564086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 843/1702] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:01.568215 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524192-7c80-478d-ba6f-d656485a9fa1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.578322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524192-7c80-478d-ba6f-d656485a9fa1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.583152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7004a-7713-4ee0-88f8-efa46913d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.583763 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7004a-7713-4ee0-88f8-efa46913d613 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.584044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7004a-7713-4ee0-88f8-efa46913d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.584326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7004a-7713-4ee0-88f8-efa46913d613 admin admin] PATH_INFO: `/v3/projects/ea92610f7a0a402d8e3d134b343fcb01/users/c45f92baac7b44d2a0830c848b8cdd75/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.599849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7004a-7713-4ee0-88f8-efa46913d613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea92610f7a0a402d8e3d134b343fcb01, user_id=c45f92baac7b44d2a0830c848b8cdd75, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.602528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7004a-7713-4ee0-88f8-efa46913d613 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.617089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 860/1703] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:01 2022] PUT /identity/v3/projects/ea92610f7a0a402d8e3d134b343fcb01/users/c45f92baac7b44d2a0830c848b8cdd75/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:01.627771 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b69a51-b28f-4311-b147-8d2d500302b0 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.628152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b69a51-b28f-4311-b147-8d2d500302b0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.628558 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b69a51-b28f-4311-b147-8d2d500302b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.656555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-39b69a51-b28f-4311-b147-8d2d500302b0 None None] MFA Rules not processed for user `c45f92baac7b44d2a0830c848b8cdd75`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:01.657749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-39b69a51-b28f-4311-b147-8d2d500302b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:01.694774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:01.722966 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 844/1704] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:01.727358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7004a-7713-4ee0-88f8-efa46913d613 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.789538 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7004a-7713-4ee0-88f8-efa46913d613 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.794530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21ba7-73dd-4c36-b2c0-e0f4290664a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.795136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21ba7-73dd-4c36-b2c0-e0f4290664a9 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.795421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21ba7-73dd-4c36-b2c0-e0f4290664a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.795701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21ba7-73dd-4c36-b2c0-e0f4290664a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.796419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21ba7-73dd-4c36-b2c0-e0f4290664a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.798284 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21ba7-73dd-4c36-b2c0-e0f4290664a9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.825069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 861/1705] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:01 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:01.829142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b69a51-b28f-4311-b147-8d2d500302b0 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.840316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b69a51-b28f-4311-b147-8d2d500302b0 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.894970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783c8e7-6095-409e-ade3-643b58ed1d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.895565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c8e7-6095-409e-ade3-643b58ed1d29 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.895843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c8e7-6095-409e-ade3-643b58ed1d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.896112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c8e7-6095-409e-ade3-643b58ed1d29 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.896775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783c8e7-6095-409e-ade3-643b58ed1d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.898677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783c8e7-6095-409e-ade3-643b58ed1d29 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.919842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 845/1706] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:01 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:01.923568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21ba7-73dd-4c36-b2c0-e0f4290664a9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.933506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21ba7-73dd-4c36-b2c0-e0f4290664a9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.938004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbc58c-9ea7-4581-bf6d-ff4515ccc98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.938589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbc58c-9ea7-4581-bf6d-ff4515ccc98a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.938868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbc58c-9ea7-4581-bf6d-ff4515ccc98a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.939142 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbc58c-9ea7-4581-bf6d-ff4515ccc98a admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.941675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbc58c-9ea7-4581-bf6d-ff4515ccc98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:01.943301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbc58c-9ea7-4581-bf6d-ff4515ccc98a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:01.956648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 862/1707] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:01.960864 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783c8e7-6095-409e-ade3-643b58ed1d29 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:01.977560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783c8e7-6095-409e-ade3-643b58ed1d29 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:01.982412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff12543-25c6-4ec7-b1d4-d72f537d9248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:01.983079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff12543-25c6-4ec7-b1d4-d72f537d9248 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:01.983363 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff12543-25c6-4ec7-b1d4-d72f537d9248 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:01.983648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff12543-25c6-4ec7-b1d4-d72f537d9248 admin admin] PATH_INFO: `/v3/projects/ce26acd1154b4ebcbf94929982413dad/users/1c0fa6c3f9ff403ca562b0f152718963/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:01.998804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff12543-25c6-4ec7-b1d4-d72f537d9248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce26acd1154b4ebcbf94929982413dad, user_id=1c0fa6c3f9ff403ca562b0f152718963, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:02.001626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff12543-25c6-4ec7-b1d4-d72f537d9248 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:02.015037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 846/1708] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:01 2022] PUT /identity/v3/projects/ce26acd1154b4ebcbf94929982413dad/users/1c0fa6c3f9ff403ca562b0f152718963/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:02.018941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbc58c-9ea7-4581-bf6d-ff4515ccc98a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:02.029358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbc58c-9ea7-4581-bf6d-ff4515ccc98a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:02.047656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:02.089446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578153a-65eb-480b-8298-ee486fe69dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:02.090076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578153a-65eb-480b-8298-ee486fe69dcf admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:02.090410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578153a-65eb-480b-8298-ee486fe69dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:02.090635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578153a-65eb-480b-8298-ee486fe69dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:02.091265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578153a-65eb-480b-8298-ee486fe69dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:02.092895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578153a-65eb-480b-8298-ee486fe69dcf admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:02.102759 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 863/1709] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:02.106978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff12543-25c6-4ec7-b1d4-d72f537d9248 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:02.117283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff12543-25c6-4ec7-b1d4-d72f537d9248 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:02.122380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd15b63-f9ac-4fc5-87a5-f178936b8d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:02.123021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd15b63-f9ac-4fc5-87a5-f178936b8d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:02.123297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd15b63-f9ac-4fc5-87a5-f178936b8d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:02.123569 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd15b63-f9ac-4fc5-87a5-f178936b8d98 admin admin] PATH_INFO: `/v3/projects/ce26acd1154b4ebcbf94929982413dad/users/1c0fa6c3f9ff403ca562b0f152718963/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:02.125498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd15b63-f9ac-4fc5-87a5-f178936b8d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce26acd1154b4ebcbf94929982413dad, user_id=1c0fa6c3f9ff403ca562b0f152718963, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:02.128105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd15b63-f9ac-4fc5-87a5-f178936b8d98 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:02.138625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 847/1710] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:02 2022] PUT /identity/v3/projects/ce26acd1154b4ebcbf94929982413dad/users/1c0fa6c3f9ff403ca562b0f152718963/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:02.150254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3a916-6bd8-4fd6-bda3-ba81b92577a9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:02.150621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3a916-6bd8-4fd6-bda3-ba81b92577a9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:02.151178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3a916-6bd8-4fd6-bda3-ba81b92577a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:02.189751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9aa3a916-6bd8-4fd6-bda3-ba81b92577a9 None None] MFA Rules not processed for user `1c0fa6c3f9ff403ca562b0f152718963`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:02.189751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9aa3a916-6bd8-4fd6-bda3-ba81b92577a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:02.256884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 864/1711] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:02.268639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd15b63-f9ac-4fc5-87a5-f178936b8d98 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:02.270330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fcd15b63-f9ac-4fc5-87a5-f178936b8d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:02.327166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd15b63-f9ac-4fc5-87a5-f178936b8d98 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:02.330767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2222fc-838b-4427-87af-c70e0b57db17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:02.331366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2222fc-838b-4427-87af-c70e0b57db17 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:02.331640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2222fc-838b-4427-87af-c70e0b57db17 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:02.331910 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2222fc-838b-4427-87af-c70e0b57db17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:02.340786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2222fc-838b-4427-87af-c70e0b57db17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:02.342157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2222fc-838b-4427-87af-c70e0b57db17 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:02.351544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 848/1712] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:02.438502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3a916-6bd8-4fd6-bda3-ba81b92577a9 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:02.448334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3a916-6bd8-4fd6-bda3-ba81b92577a9 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:02.451884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0a74c-18d5-4ebd-b4e6-5a4c60272fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:02.452518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0a74c-18d5-4ebd-b4e6-5a4c60272fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:02.452817 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0a74c-18d5-4ebd-b4e6-5a4c60272fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:02.453109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0a74c-18d5-4ebd-b4e6-5a4c60272fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:02.508777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0a74c-18d5-4ebd-b4e6-5a4c60272fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:02.510196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0a74c-18d5-4ebd-b4e6-5a4c60272fd1 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:02.518972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 865/1713] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:05.367102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2222fc-838b-4427-87af-c70e0b57db17 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:05.368652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ce2222fc-838b-4427-87af-c70e0b57db17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:05.381000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2222fc-838b-4427-87af-c70e0b57db17 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:05.384656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446225d-b7a3-4361-895b-421fff458d01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:05.385301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446225d-b7a3-4361-895b-421fff458d01 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:05.385574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446225d-b7a3-4361-895b-421fff458d01 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:05.385867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446225d-b7a3-4361-895b-421fff458d01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:05.387665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0446225d-b7a3-4361-895b-421fff458d01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:05.427268 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:05.464141 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446225d-b7a3-4361-895b-421fff458d01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:05.465722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446225d-b7a3-4361-895b-421fff458d01 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:05.476176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 849/1714] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:08.643776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0a74c-18d5-4ebd-b4e6-5a4c60272fd1 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:08.697217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:08.718169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0a74c-18d5-4ebd-b4e6-5a4c60272fd1 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:08.722754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:08.723370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:08.723646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:08.723917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] PATH_INFO: `/v3/users/c45f92baac7b44d2a0830c848b8cdd75` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:08.725037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45f92baac7b44d2a0830c848b8cdd75)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:08.726933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:08.781467 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45f92baac7b44d2a0830c848b8cdd75'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:08.792039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45f92baac7b44d2a0830c848b8cdd75'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:08.792743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45f92baac7b44d2a0830c848b8cdd75'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:08.794226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45f92baac7b44d2a0830c848b8cdd75'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:08.804586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45f92baac7b44d2a0830c848b8cdd75'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:08.823725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 866/1715] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:08 2022] DELETE /identity/v3/users/c45f92baac7b44d2a0830c848b8cdd75 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:08.828593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405f0dc-d570-4cf5-8b94-f29e67eb556c None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:08.829114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405f0dc-d570-4cf5-8b94-f29e67eb556c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:08.829219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405f0dc-d570-4cf5-8b94-f29e67eb556c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:08.851856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-9405f0dc-d570-4cf5-8b94-f29e67eb556c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:08.853384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9405f0dc-d570-4cf5-8b94-f29e67eb556c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:08.924296 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 850/1716] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:08.934597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:08.936376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:09.008583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952cbbc-9bbe-43f1-ab33-3161f362fee9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:09.021229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d8cdd-9c13-41f8-89c1-2ac2d755f5b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:09.024453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d8cdd-9c13-41f8-89c1-2ac2d755f5b5 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:09.025736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d8cdd-9c13-41f8-89c1-2ac2d755f5b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:09.026385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d8cdd-9c13-41f8-89c1-2ac2d755f5b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:09.039860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d8cdd-9c13-41f8-89c1-2ac2d755f5b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:09.041326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d8cdd-9c13-41f8-89c1-2ac2d755f5b5 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:09.055275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 867/1717] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:09.110236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405f0dc-d570-4cf5-8b94-f29e67eb556c None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:09.112021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9405f0dc-d570-4cf5-8b94-f29e67eb556c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:09.125338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405f0dc-d570-4cf5-8b94-f29e67eb556c None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:09.131512 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:09.132304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:09.132714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:09.133096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] PATH_INFO: `/v3/users/06a10bcd35d34d1e922b30db9708f297` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:09.134779 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a10bcd35d34d1e922b30db9708f297)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:09.137465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:09.202656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a10bcd35d34d1e922b30db9708f297'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.226211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a10bcd35d34d1e922b30db9708f297'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.227021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a10bcd35d34d1e922b30db9708f297'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.227693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a10bcd35d34d1e922b30db9708f297'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.254899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a10bcd35d34d1e922b30db9708f297'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.275354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 851/1718] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:09 2022] DELETE /identity/v3/users/06a10bcd35d34d1e922b30db9708f297 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:09.280088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557735ab-e177-4772-8e2f-4655dc6190bd None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:09.280650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557735ab-e177-4772-8e2f-4655dc6190bd None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:09.280923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557735ab-e177-4772-8e2f-4655dc6190bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:09.301745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-557735ab-e177-4772-8e2f-4655dc6190bd None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:09.302967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-557735ab-e177-4772-8e2f-4655dc6190bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:09.382007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:09.384096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:09.385136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 868/1719] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:42:09.400900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557735ab-e177-4772-8e2f-4655dc6190bd None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:09.403076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-557735ab-e177-4772-8e2f-4655dc6190bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:09.409366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:09.412650 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bebf0-4840-4f5b-9251-75a0f7d51ade admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:09.421322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:09.422112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:09.422489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:09.422896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] PATH_INFO: `/v3/projects/ea92610f7a0a402d8e3d134b343fcb01` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:09.424192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea92610f7a0a402d8e3d134b343fcb01)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:09.426623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:09.476287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:09.490532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557735ab-e177-4772-8e2f-4655dc6190bd None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:09.491403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea92610f7a0a402d8e3d134b343fcb01', 'request_id': 'req-51073bfa-ee90-43b9-b9a1-d44f582e44b3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.500249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea92610f7a0a402d8e3d134b343fcb01', 'request_id': 'req-51073bfa-ee90-43b9-b9a1-d44f582e44b3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.510430 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea92610f7a0a402d8e3d134b343fcb01', 'request_id': 'req-51073bfa-ee90-43b9-b9a1-d44f582e44b3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.524545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] The token cache is being invalidate because project ea92610f7a0a402d8e3d134b343fcb01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:09.524968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.525958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.528506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 852/1720] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:09 2022] DELETE /identity/v3/projects/ea92610f7a0a402d8e3d134b343fcb01 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:09.532605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:09.534348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:09.546057 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:09.548166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:09.548686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:09.550085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:09.553297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:09.625284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073bfa-ee90-43b9-b9a1-d44f582e44b3 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:09.630103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:09.630813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:09.631161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:09.631625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] PATH_INFO: `/v3/users/1c0fa6c3f9ff403ca562b0f152718963` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:09.632861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0fa6c3f9ff403ca562b0f152718963)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:09.634867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:09.639512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:09.642348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:09.656083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 869/1721] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:09.706660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0fa6c3f9ff403ca562b0f152718963'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.714210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0fa6c3f9ff403ca562b0f152718963'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.715714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0fa6c3f9ff403ca562b0f152718963'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.716483 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0fa6c3f9ff403ca562b0f152718963'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.728643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0fa6c3f9ff403ca562b0f152718963'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:09.763938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 853/1722] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:09 2022] DELETE /identity/v3/users/1c0fa6c3f9ff403ca562b0f152718963 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:09.966808 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:09.968268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:09.981004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deaf463-2a81-4ef2-aa09-ad734bb523e4 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.047422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.048135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.048533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.048860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] PATH_INFO: `/v3/projects/7ab47d6fe38b45dbb97fe34b2dacac6b` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.050234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab47d6fe38b45dbb97fe34b2dacac6b)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.060823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.062836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:10.073994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.089302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:10.107858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb38e8-06aa-49e5-9159-4490b2b4e2de admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.129910 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.130616 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.131127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.131410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] PATH_INFO: `/v3/projects/ce26acd1154b4ebcbf94929982413dad` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.132728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce26acd1154b4ebcbf94929982413dad)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.136249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.157102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab47d6fe38b45dbb97fe34b2dacac6b', 'request_id': 'req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.163122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab47d6fe38b45dbb97fe34b2dacac6b', 'request_id': 'req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.173235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab47d6fe38b45dbb97fe34b2dacac6b', 'request_id': 'req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.188972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:10.198767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] The token cache is being invalidate because project 7ab47d6fe38b45dbb97fe34b2dacac6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:10.199911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.200888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.202522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce26acd1154b4ebcbf94929982413dad', 'request_id': 'req-b0a9ac00-9de0-4a63-93b7-5500361b7385'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.203122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 870/1723] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:09 2022] DELETE /identity/v3/projects/7ab47d6fe38b45dbb97fe34b2dacac6b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:10.206755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.208129 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:10.209633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce26acd1154b4ebcbf94929982413dad', 'request_id': 'req-b0a9ac00-9de0-4a63-93b7-5500361b7385'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.219512 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce26acd1154b4ebcbf94929982413dad', 'request_id': 'req-b0a9ac00-9de0-4a63-93b7-5500361b7385'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.232629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] The token cache is being invalidate because project ce26acd1154b4ebcbf94929982413dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:10.233004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.233726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b0a9ac00-9de0-4a63-93b7-5500361b7385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.238925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 854/1724] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:10 2022] DELETE /identity/v3/projects/ce26acd1154b4ebcbf94929982413dad => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:10.241976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebaf4c-1546-4d38-9687-3323b251c006 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.242444 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebaf4c-1546-4d38-9687-3323b251c006 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.242853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebaf4c-1546-4d38-9687-3323b251c006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.265543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d7ebaf4c-1546-4d38-9687-3323b251c006 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:10.266905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d7ebaf4c-1546-4d38-9687-3323b251c006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:10.281424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf32ea6-129d-4f76-bbe6-b7582ba871d5 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.347619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 855/1725] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:10.356075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14977b6c-66ee-4b6b-8aab-7832f0f695dc None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.356075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14977b6c-66ee-4b6b-8aab-7832f0f695dc None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.356479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14977b6c-66ee-4b6b-8aab-7832f0f695dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.368744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.369581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.369984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.370262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] PATH_INFO: `/v3/users/47fd2100eb304efa80276f2d600689af` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.372293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] RBAC: Authorizing `identity:delete_user(user_id=47fd2100eb304efa80276f2d600689af)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.374497 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.382299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-14977b6c-66ee-4b6b-8aab-7832f0f695dc None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:10.384133 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-14977b6c-66ee-4b6b-8aab-7832f0f695dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:10.396632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 856/1726] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:42:10.399315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14977b6c-66ee-4b6b-8aab-7832f0f695dc None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.410794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14977b6c-66ee-4b6b-8aab-7832f0f695dc None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.417297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c704f5-778c-4daf-96b2-d08beb211ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.417989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c704f5-778c-4daf-96b2-d08beb211ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.418252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c704f5-778c-4daf-96b2-d08beb211ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.418529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c704f5-778c-4daf-96b2-d08beb211ef5 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.419387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c704f5-778c-4daf-96b2-d08beb211ef5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:42:10.419800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c704f5-778c-4daf-96b2-d08beb211ef5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.421960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c704f5-778c-4daf-96b2-d08beb211ef5 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.437710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 857/1727] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:42:10.442285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c704f5-778c-4daf-96b2-d08beb211ef5 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.445773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47fd2100eb304efa80276f2d600689af'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.447004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:10.456791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47fd2100eb304efa80276f2d600689af'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.458204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c704f5-778c-4daf-96b2-d08beb211ef5 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.458413 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47fd2100eb304efa80276f2d600689af'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.459342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47fd2100eb304efa80276f2d600689af'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.465034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc36a3-302f-4b1b-8c74-65524fe05378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.465720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc36a3-302f-4b1b-8c74-65524fe05378 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.466000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc36a3-302f-4b1b-8c74-65524fe05378 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.467666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc36a3-302f-4b1b-8c74-65524fe05378 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.468179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47fd2100eb304efa80276f2d600689af'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:10.468315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc36a3-302f-4b1b-8c74-65524fe05378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.470208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc36a3-302f-4b1b-8c74-65524fe05378 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.493377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 871/1728] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:10 2022] DELETE /identity/v3/users/47fd2100eb304efa80276f2d600689af => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:10.508410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 858/1729] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:10 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:10.511157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.513589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:10.581804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e1a1-c673-419b-a4c3-5b67283fa53f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.642994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea256df-4bfa-420a-8cf5-59195db0f4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.643433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea256df-4bfa-420a-8cf5-59195db0f4bc admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.643716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea256df-4bfa-420a-8cf5-59195db0f4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.643993 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea256df-4bfa-420a-8cf5-59195db0f4bc admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.644677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea256df-4bfa-420a-8cf5-59195db0f4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.646776 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea256df-4bfa-420a-8cf5-59195db0f4bc admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.672754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 872/1730] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:10 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:10.677073 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc36a3-302f-4b1b-8c74-65524fe05378 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.688074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc36a3-302f-4b1b-8c74-65524fe05378 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.693305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f91d08-6b68-4152-b270-7a21f6d2b592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.693922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f91d08-6b68-4152-b270-7a21f6d2b592 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.694202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f91d08-6b68-4152-b270-7a21f6d2b592 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.694476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f91d08-6b68-4152-b270-7a21f6d2b592 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.695120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f91d08-6b68-4152-b270-7a21f6d2b592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.696869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f91d08-6b68-4152-b270-7a21f6d2b592 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.711585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 859/1731] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:10.716173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea256df-4bfa-420a-8cf5-59195db0f4bc admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.727113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea256df-4bfa-420a-8cf5-59195db0f4bc admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.732651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201a4d5-75eb-41ed-85c7-3c265f570e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.733412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201a4d5-75eb-41ed-85c7-3c265f570e0a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.733768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201a4d5-75eb-41ed-85c7-3c265f570e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.734118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201a4d5-75eb-41ed-85c7-3c265f570e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.735088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201a4d5-75eb-41ed-85c7-3c265f570e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.736948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201a4d5-75eb-41ed-85c7-3c265f570e0a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.747553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 873/1732] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:10.751817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f91d08-6b68-4152-b270-7a21f6d2b592 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.764270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f91d08-6b68-4152-b270-7a21f6d2b592 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.771293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44444a-dc60-41c1-b1bf-1eccf6a22c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.771900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44444a-dc60-41c1-b1bf-1eccf6a22c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.772262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44444a-dc60-41c1-b1bf-1eccf6a22c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.772592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44444a-dc60-41c1-b1bf-1eccf6a22c0f admin admin] PATH_INFO: `/v3/projects/af3380a2571c4631a1037faded5f912c/users/2ed54c246a8548dd87b6f806e8e7ac3c/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.793822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44444a-dc60-41c1-b1bf-1eccf6a22c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3380a2571c4631a1037faded5f912c, user_id=2ed54c246a8548dd87b6f806e8e7ac3c, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.798665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201a4d5-75eb-41ed-85c7-3c265f570e0a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.798815 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44444a-dc60-41c1-b1bf-1eccf6a22c0f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.801409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6201a4d5-75eb-41ed-85c7-3c265f570e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:10.832230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201a4d5-75eb-41ed-85c7-3c265f570e0a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.836394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 860/1733] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:10 2022] PUT /identity/v3/projects/af3380a2571c4631a1037faded5f912c/users/2ed54c246a8548dd87b6f806e8e7ac3c/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:10.847188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e573a-a27b-4b52-a4e0-146408032dc5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.847488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e573a-a27b-4b52-a4e0-146408032dc5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.847751 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e573a-a27b-4b52-a4e0-146408032dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.881529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-734e573a-a27b-4b52-a4e0-146408032dc5 None None] MFA Rules not processed for user `2ed54c246a8548dd87b6f806e8e7ac3c`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:10.882829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-734e573a-a27b-4b52-a4e0-146408032dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:10.910264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:10.925212 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.925845 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.926132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.926412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] PATH_INFO: `/v3/projects/9708996875cd45ca885b249b335a24fe` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.927554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9708996875cd45ca885b249b335a24fe)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.928470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:10.929627 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:10.960330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 861/1734] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:10.964826 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e573a-a27b-4b52-a4e0-146408032dc5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:10.976753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e573a-a27b-4b52-a4e0-146408032dc5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:10.983300 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07e213-994f-4916-9f05-8c917d526446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:10.983925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07e213-994f-4916-9f05-8c917d526446 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:10.984195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07e213-994f-4916-9f05-8c917d526446 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:10.984571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07e213-994f-4916-9f05-8c917d526446 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:10.985581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07e213-994f-4916-9f05-8c917d526446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:10.987809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07e213-994f-4916-9f05-8c917d526446 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.000906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9708996875cd45ca885b249b335a24fe', 'request_id': 'req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:11.006893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9708996875cd45ca885b249b335a24fe', 'request_id': 'req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:11.015158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9708996875cd45ca885b249b335a24fe', 'request_id': 'req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:11.018509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 862/1735] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:10 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:11.022455 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07e213-994f-4916-9f05-8c917d526446 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.028891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] The token cache is being invalidate because project 9708996875cd45ca885b249b335a24fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:11.029034 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:11.029769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cd7331cb-df4f-4b77-bfd5-38b20c0f78dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:11.031376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 874/1736] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:10 2022] DELETE /identity/v3/projects/9708996875cd45ca885b249b335a24fe => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:11.036879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c37a6-3ee9-4192-8af1-5ecea7f776ae None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.037297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c37a6-3ee9-4192-8af1-5ecea7f776ae None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.037733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c37a6-3ee9-4192-8af1-5ecea7f776ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.064144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d71c37a6-3ee9-4192-8af1-5ecea7f776ae None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:11.065897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d71c37a6-3ee9-4192-8af1-5ecea7f776ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:11.094109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07e213-994f-4916-9f05-8c917d526446 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.099066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1711cf0-b751-4c4a-a074-92555e534a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.099673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1711cf0-b751-4c4a-a074-92555e534a70 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.099956 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1711cf0-b751-4c4a-a074-92555e534a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.100231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1711cf0-b751-4c4a-a074-92555e534a70 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.100896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1711cf0-b751-4c4a-a074-92555e534a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.102830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1711cf0-b751-4c4a-a074-92555e534a70 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.127395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 863/1737] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:11 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:11.130068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1711cf0-b751-4c4a-a074-92555e534a70 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.131782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f1711cf0-b751-4c4a-a074-92555e534a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:11.135353 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 875/1738] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:11.143587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941dd5a-525b-477b-9926-5d2c0e0c5492 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.143880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941dd5a-525b-477b-9926-5d2c0e0c5492 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.144153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941dd5a-525b-477b-9926-5d2c0e0c5492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.145059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1711cf0-b751-4c4a-a074-92555e534a70 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.149694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb72941-c51b-436e-aab3-2154f99a74e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.150365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb72941-c51b-436e-aab3-2154f99a74e3 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.150658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb72941-c51b-436e-aab3-2154f99a74e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.150993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb72941-c51b-436e-aab3-2154f99a74e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.151665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb72941-c51b-436e-aab3-2154f99a74e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.153320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb72941-c51b-436e-aab3-2154f99a74e3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.165583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9941dd5a-525b-477b-9926-5d2c0e0c5492 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:11.166808 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9941dd5a-525b-477b-9926-5d2c0e0c5492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:11.167337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 864/1739] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:11.171275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb72941-c51b-436e-aab3-2154f99a74e3 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.186187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb72941-c51b-436e-aab3-2154f99a74e3 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.187303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 876/1740] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:11.191321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941dd5a-525b-477b-9926-5d2c0e0c5492 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.191448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13d621-8e5a-4fba-a97d-ed48d44b822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.192003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13d621-8e5a-4fba-a97d-ed48d44b822b admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.192288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13d621-8e5a-4fba-a97d-ed48d44b822b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.192593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13d621-8e5a-4fba-a97d-ed48d44b822b admin admin] PATH_INFO: `/v3/projects/1e81592f70ab4e219c04f9934e2eb3f8/users/548c3e05cd674246ad287590c2e4193c/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.200428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941dd5a-525b-477b-9926-5d2c0e0c5492 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.205232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5afba-7923-4991-bed7-1fd8ecb1836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.205754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5afba-7923-4991-bed7-1fd8ecb1836a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.206037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5afba-7923-4991-bed7-1fd8ecb1836a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.206317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5afba-7923-4991-bed7-1fd8ecb1836a admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.206968 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5afba-7923-4991-bed7-1fd8ecb1836a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:42:11.207275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5afba-7923-4991-bed7-1fd8ecb1836a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.207755 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13d621-8e5a-4fba-a97d-ed48d44b822b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e81592f70ab4e219c04f9934e2eb3f8, user_id=548c3e05cd674246ad287590c2e4193c, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.208899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5afba-7923-4991-bed7-1fd8ecb1836a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.210488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13d621-8e5a-4fba-a97d-ed48d44b822b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.222261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 877/1741] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:42:11.226169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 865/1742] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:11 2022] PUT /identity/v3/projects/1e81592f70ab4e219c04f9934e2eb3f8/users/548c3e05cd674246ad287590c2e4193c/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:11.226412 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5afba-7923-4991-bed7-1fd8ecb1836a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.230093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13d621-8e5a-4fba-a97d-ed48d44b822b admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.236877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5afba-7923-4991-bed7-1fd8ecb1836a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.301860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13d621-8e5a-4fba-a97d-ed48d44b822b admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.306694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fbfae-a9b4-4ca8-8e7b-564ee60fce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.307300 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fbfae-a9b4-4ca8-8e7b-564ee60fce21 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.307580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fbfae-a9b4-4ca8-8e7b-564ee60fce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.307856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fbfae-a9b4-4ca8-8e7b-564ee60fce21 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.309042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fbfae-a9b4-4ca8-8e7b-564ee60fce21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.310531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aeaab1-d145-41b8-a589-991fcd583ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.310741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fbfae-a9b4-4ca8-8e7b-564ee60fce21 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.311312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aeaab1-d145-41b8-a589-991fcd583ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.311750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aeaab1-d145-41b8-a589-991fcd583ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.312083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aeaab1-d145-41b8-a589-991fcd583ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.312891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aeaab1-d145-41b8-a589-991fcd583ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.315055 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aeaab1-d145-41b8-a589-991fcd583ea7 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.321351 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:11.322002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 866/1743] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:11.326967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fbfae-a9b4-4ca8-8e7b-564ee60fce21 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.336854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fbfae-a9b4-4ca8-8e7b-564ee60fce21 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.346709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7ebf0-33eb-475e-9681-1b5a2f03aaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.347369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7ebf0-33eb-475e-9681-1b5a2f03aaf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.347652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7ebf0-33eb-475e-9681-1b5a2f03aaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.347932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7ebf0-33eb-475e-9681-1b5a2f03aaf7 admin admin] PATH_INFO: `/v3/projects/1e81592f70ab4e219c04f9934e2eb3f8/users/548c3e05cd674246ad287590c2e4193c/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.353529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 878/1744] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:11 2022] POST /identity/v3/projects => generated 389 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:11.354956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aeaab1-d145-41b8-a589-991fcd583ea7 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.357473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7ebf0-33eb-475e-9681-1b5a2f03aaf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e81592f70ab4e219c04f9934e2eb3f8, user_id=548c3e05cd674246ad287590c2e4193c, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.360457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7ebf0-33eb-475e-9681-1b5a2f03aaf7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.362961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:11.373467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 867/1745] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:11 2022] PUT /identity/v3/projects/1e81592f70ab4e219c04f9934e2eb3f8/users/548c3e05cd674246ad287590c2e4193c/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:11.383629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bb710-5e89-4f20-869f-1ae4cfc4f366 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.383923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bb710-5e89-4f20-869f-1ae4cfc4f366 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.384189 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bb710-5e89-4f20-869f-1ae4cfc4f366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.411574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-0d0bb710-5e89-4f20-869f-1ae4cfc4f366 None None] MFA Rules not processed for user `548c3e05cd674246ad287590c2e4193c`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:11.412723 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0d0bb710-5e89-4f20-869f-1ae4cfc4f366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:11.420633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aeaab1-d145-41b8-a589-991fcd583ea7 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.476094 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50568220-4cfa-461c-a64f-c478d4838264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.476735 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50568220-4cfa-461c-a64f-c478d4838264 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.477012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50568220-4cfa-461c-a64f-c478d4838264 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.477282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50568220-4cfa-461c-a64f-c478d4838264 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.477937 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50568220-4cfa-461c-a64f-c478d4838264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.479837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50568220-4cfa-461c-a64f-c478d4838264 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.482095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 868/1746] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:42:11.496073 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bb710-5e89-4f20-869f-1ae4cfc4f366 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.497523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0d0bb710-5e89-4f20-869f-1ae4cfc4f366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:11.502586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 879/1747] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:11 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:11.506428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50568220-4cfa-461c-a64f-c478d4838264 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.517333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50568220-4cfa-461c-a64f-c478d4838264 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.525522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c33f7a-ecd6-444d-ab3a-2b8d7a2f31cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.526133 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c33f7a-ecd6-444d-ab3a-2b8d7a2f31cf admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.526419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c33f7a-ecd6-444d-ab3a-2b8d7a2f31cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.526687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c33f7a-ecd6-444d-ab3a-2b8d7a2f31cf admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.527874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c33f7a-ecd6-444d-ab3a-2b8d7a2f31cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.529815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c33f7a-ecd6-444d-ab3a-2b8d7a2f31cf admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.545657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 880/1748] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:11.547546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c33f7a-ecd6-444d-ab3a-2b8d7a2f31cf admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.558583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c33f7a-ecd6-444d-ab3a-2b8d7a2f31cf admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.563336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1e570-97a5-4326-ab47-fb1cecb1732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.563946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1e570-97a5-4326-ab47-fb1cecb1732e admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.564229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1e570-97a5-4326-ab47-fb1cecb1732e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.564968 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1e570-97a5-4326-ab47-fb1cecb1732e admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.565140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bb710-5e89-4f20-869f-1ae4cfc4f366 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.565462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e1e570-97a5-4326-ab47-fb1cecb1732e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.567036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e1e570-97a5-4326-ab47-fb1cecb1732e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.568950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492abb1-0b2b-480d-9fb4-30669c144117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.569904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492abb1-0b2b-480d-9fb4-30669c144117 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.570183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492abb1-0b2b-480d-9fb4-30669c144117 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.570459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492abb1-0b2b-480d-9fb4-30669c144117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.581448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 881/1749] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:11.584854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492abb1-0b2b-480d-9fb4-30669c144117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.585695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1e570-97a5-4326-ab47-fb1cecb1732e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.586312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492abb1-0b2b-480d-9fb4-30669c144117 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.596556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1e570-97a5-4326-ab47-fb1cecb1732e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.601234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf70ca9-ea76-460d-9e59-2d226d117a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.601881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf70ca9-ea76-460d-9e59-2d226d117a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.602158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf70ca9-ea76-460d-9e59-2d226d117a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.602428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf70ca9-ea76-460d-9e59-2d226d117a7d admin admin] PATH_INFO: `/v3/projects/6d98f052e0bf46098105ed623fa90c20/users/9741a138bcd546428f0214ea30f7217b/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.608710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 869/1750] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:42:11.620194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf70ca9-ea76-460d-9e59-2d226d117a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d98f052e0bf46098105ed623fa90c20, user_id=9741a138bcd546428f0214ea30f7217b, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.623941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf70ca9-ea76-460d-9e59-2d226d117a7d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.654683 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 882/1751] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:11 2022] PUT /identity/v3/projects/6d98f052e0bf46098105ed623fa90c20/users/9741a138bcd546428f0214ea30f7217b/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:11.665236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8572456-24a1-4910-82a4-e017f4d060a5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.665706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8572456-24a1-4910-82a4-e017f4d060a5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.665984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8572456-24a1-4910-82a4-e017f4d060a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.702457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf70ca9-ea76-460d-9e59-2d226d117a7d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.704677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-e8572456-24a1-4910-82a4-e017f4d060a5 None None] MFA Rules not processed for user `9741a138bcd546428f0214ea30f7217b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:11.706238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e8572456-24a1-4910-82a4-e017f4d060a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:11.748995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:11.766245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf70ca9-ea76-460d-9e59-2d226d117a7d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.770285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7d237-d684-4ed3-955c-28aa2c83fe99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.770897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7d237-d684-4ed3-955c-28aa2c83fe99 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.771174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7d237-d684-4ed3-955c-28aa2c83fe99 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.771446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7d237-d684-4ed3-955c-28aa2c83fe99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.773599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-51e7d237-d684-4ed3-955c-28aa2c83fe99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:11.782369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 870/1752] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:11.787705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8572456-24a1-4910-82a4-e017f4d060a5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.802655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:11.841370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7d237-d684-4ed3-955c-28aa2c83fe99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.843160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7d237-d684-4ed3-955c-28aa2c83fe99 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.853252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 883/1753] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:42:11.866448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8572456-24a1-4910-82a4-e017f4d060a5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:11.871603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fa374-d4a0-4a69-b0ed-8fd7834a8478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:11.872213 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fa374-d4a0-4a69-b0ed-8fd7834a8478 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:11.872529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fa374-d4a0-4a69-b0ed-8fd7834a8478 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:11.872799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fa374-d4a0-4a69-b0ed-8fd7834a8478 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:11.873429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fa374-d4a0-4a69-b0ed-8fd7834a8478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:11.875690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fa374-d4a0-4a69-b0ed-8fd7834a8478 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:11.910145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 871/1754] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:11 2022] POST /identity/v3/projects => generated 389 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:11.914123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7d237-d684-4ed3-955c-28aa2c83fe99 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:11.926887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7d237-d684-4ed3-955c-28aa2c83fe99 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:12.006323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c737e00-fde8-44ce-ba42-decabe072be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:12.006957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c737e00-fde8-44ce-ba42-decabe072be9 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.007224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c737e00-fde8-44ce-ba42-decabe072be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.007501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c737e00-fde8-44ce-ba42-decabe072be9 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.008144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c737e00-fde8-44ce-ba42-decabe072be9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:12.010102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c737e00-fde8-44ce-ba42-decabe072be9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:12.042988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 884/1755] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:11 2022] POST /identity/v3/users => generated 301 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:12.044705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fa374-d4a0-4a69-b0ed-8fd7834a8478 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:12.058114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fa374-d4a0-4a69-b0ed-8fd7834a8478 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:12.064259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baa67f-a98d-48da-93d5-f7f881fe7e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:12.064682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baa67f-a98d-48da-93d5-f7f881fe7e5c admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.064966 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baa67f-a98d-48da-93d5-f7f881fe7e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.065230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baa67f-a98d-48da-93d5-f7f881fe7e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.065879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baa67f-a98d-48da-93d5-f7f881fe7e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:12.067619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baa67f-a98d-48da-93d5-f7f881fe7e5c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:12.083803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 872/1756] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:12.085171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c737e00-fde8-44ce-ba42-decabe072be9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:12.104250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c737e00-fde8-44ce-ba42-decabe072be9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:12.113120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7c2f-c54d-4b8b-99f2-6fc604e90ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:12.117971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7c2f-c54d-4b8b-99f2-6fc604e90ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.120192 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7c2f-c54d-4b8b-99f2-6fc604e90ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.120705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7c2f-c54d-4b8b-99f2-6fc604e90ce0 admin admin] PATH_INFO: `/v3/projects/ea451ea7820a45f99b254dbc136d1d39/users/2302068cc499439aa1ca6af1a5f4e33f/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.141175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc7c2f-c54d-4b8b-99f2-6fc604e90ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea451ea7820a45f99b254dbc136d1d39, user_id=2302068cc499439aa1ca6af1a5f4e33f, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:12.144863 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc7c2f-c54d-4b8b-99f2-6fc604e90ce0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:12.165561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 885/1757] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:12 2022] PUT /identity/v3/projects/ea451ea7820a45f99b254dbc136d1d39/users/2302068cc499439aa1ca6af1a5f4e33f/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:12.170304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baa67f-a98d-48da-93d5-f7f881fe7e5c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:12.188210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baa67f-a98d-48da-93d5-f7f881fe7e5c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:12.189213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7c2f-c54d-4b8b-99f2-6fc604e90ce0 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:12.284982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:12.289975 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7c2f-c54d-4b8b-99f2-6fc604e90ce0 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:12.303149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:12.303799 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.304081 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.305038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] PATH_INFO: `/v3/users/2ed54c246a8548dd87b6f806e8e7ac3c` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.306169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed54c246a8548dd87b6f806e8e7ac3c)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:12.308321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07fb07-043c-4abc-95ca-41e1bb924600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:12.309754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07fb07-043c-4abc-95ca-41e1bb924600 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.310037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07fb07-043c-4abc-95ca-41e1bb924600 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.310317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07fb07-043c-4abc-95ca-41e1bb924600 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.310963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07fb07-043c-4abc-95ca-41e1bb924600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:12.312583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07fb07-043c-4abc-95ca-41e1bb924600 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:12.314959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:12.328837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 873/1758] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:12.333320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07fb07-043c-4abc-95ca-41e1bb924600 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:12.360101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07fb07-043c-4abc-95ca-41e1bb924600 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:12.370822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf079f-8300-4a4c-8663-eff7ff817b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:12.372069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf079f-8300-4a4c-8663-eff7ff817b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.372554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf079f-8300-4a4c-8663-eff7ff817b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.373120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf079f-8300-4a4c-8663-eff7ff817b02 admin admin] PATH_INFO: `/v3/projects/ea451ea7820a45f99b254dbc136d1d39/users/2302068cc499439aa1ca6af1a5f4e33f/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.373734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed54c246a8548dd87b6f806e8e7ac3c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:12.376374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf079f-8300-4a4c-8663-eff7ff817b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea451ea7820a45f99b254dbc136d1d39, user_id=2302068cc499439aa1ca6af1a5f4e33f, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:12.380742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf079f-8300-4a4c-8663-eff7ff817b02 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:12.382469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed54c246a8548dd87b6f806e8e7ac3c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:12.383417 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed54c246a8548dd87b6f806e8e7ac3c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:12.384188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed54c246a8548dd87b6f806e8e7ac3c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:12.395068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 874/1759] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:12 2022] PUT /identity/v3/projects/ea451ea7820a45f99b254dbc136d1d39/users/2302068cc499439aa1ca6af1a5f4e33f/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:12.395788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-86ee95a7-dc2e-4017-a314-7322e4d09591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed54c246a8548dd87b6f806e8e7ac3c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:12.406707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfdd93-ef19-49ed-a2ed-a2627b2bd698 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.407522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfdd93-ef19-49ed-a2ed-a2627b2bd698 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.407879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfdd93-ef19-49ed-a2ed-a2627b2bd698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.422875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 886/1760] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:12 2022] DELETE /identity/v3/users/2ed54c246a8548dd87b6f806e8e7ac3c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:12.428141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d53041-dddc-436a-b3cb-40e58e02e43f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.428447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d53041-dddc-436a-b3cb-40e58e02e43f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.428682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d53041-dddc-436a-b3cb-40e58e02e43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.445016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6abfdd93-ef19-49ed-a2ed-a2627b2bd698 None None] MFA Rules not processed for user `2302068cc499439aa1ca6af1a5f4e33f`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:12.446517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6abfdd93-ef19-49ed-a2ed-a2627b2bd698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:12.467059 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-92d53041-dddc-436a-b3cb-40e58e02e43f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:12.468578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-92d53041-dddc-436a-b3cb-40e58e02e43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:12.570827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 887/1761] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:12.581982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d53041-dddc-436a-b3cb-40e58e02e43f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:12.583549 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-92d53041-dddc-436a-b3cb-40e58e02e43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:12.586454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:12.606668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 875/1762] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:12.618848 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfdd93-ef19-49ed-a2ed-a2627b2bd698 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:12.620426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6abfdd93-ef19-49ed-a2ed-a2627b2bd698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:12.661739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d53041-dddc-436a-b3cb-40e58e02e43f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:12.665935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc6c9b-8ada-4f79-ae9e-f79664ec8bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:12.666511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6c9b-8ada-4f79-ae9e-f79664ec8bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.666848 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6c9b-8ada-4f79-ae9e-f79664ec8bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.667057 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6c9b-8ada-4f79-ae9e-f79664ec8bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.677021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc6c9b-8ada-4f79-ae9e-f79664ec8bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:12.679272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc6c9b-8ada-4f79-ae9e-f79664ec8bb7 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:12.688796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfdd93-ef19-49ed-a2ed-a2627b2bd698 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:12.692230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 888/1763] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:42:12.692795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b91e4d-21ba-4e1b-ab61-3a52ac94512b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:12.693417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b91e4d-21ba-4e1b-ab61-3a52ac94512b service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.693701 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b91e4d-21ba-4e1b-ab61-3a52ac94512b service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.694000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b91e4d-21ba-4e1b-ab61-3a52ac94512b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.704640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b91e4d-21ba-4e1b-ab61-3a52ac94512b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:12.706336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b91e4d-21ba-4e1b-ab61-3a52ac94512b service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:12.718959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 876/1764] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:12.867141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc6c9b-8ada-4f79-ae9e-f79664ec8bb7 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:12.868915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c5fc6c9b-8ada-4f79-ae9e-f79664ec8bb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:12.884201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc6c9b-8ada-4f79-ae9e-f79664ec8bb7 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:12.891141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:12.891945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:12.892656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:12.892656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] PATH_INFO: `/v3/users/9741a138bcd546428f0214ea30f7217b` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:12.894138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9741a138bcd546428f0214ea30f7217b)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:12.896186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:12.918333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:12.969883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9741a138bcd546428f0214ea30f7217b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:12.978090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9741a138bcd546428f0214ea30f7217b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:12.978815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9741a138bcd546428f0214ea30f7217b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:12.979442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9741a138bcd546428f0214ea30f7217b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:12.988263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9741a138bcd546428f0214ea30f7217b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.012058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 889/1765] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:12 2022] DELETE /identity/v3/users/9741a138bcd546428f0214ea30f7217b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:13.016838 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de0ba6-36c4-4d79-b817-63c75e21af40 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:13.017180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de0ba6-36c4-4d79-b817-63c75e21af40 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:13.017487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de0ba6-36c4-4d79-b817-63c75e21af40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:13.022202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:13.024223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:13.063595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-e2de0ba6-36c4-4d79-b817-63c75e21af40 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:13.065614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e2de0ba6-36c4-4d79-b817-63c75e21af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:13.132195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c8d59-40f2-4745-b477-bb8c3eb0632c admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:13.138189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:13.138838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:13.139116 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:13.139393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] PATH_INFO: `/v3/projects/af3380a2571c4631a1037faded5f912c` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:13.141216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3380a2571c4631a1037faded5f912c)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:13.143132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:13.162860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 877/1766] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:42:13.169784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de0ba6-36c4-4d79-b817-63c75e21af40 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:13.171805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e2de0ba6-36c4-4d79-b817-63c75e21af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:13.206484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3380a2571c4631a1037faded5f912c', 'request_id': 'req-4265e3a9-f8a8-4296-8238-b525b405da44'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.217038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3380a2571c4631a1037faded5f912c', 'request_id': 'req-4265e3a9-f8a8-4296-8238-b525b405da44'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.235138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3380a2571c4631a1037faded5f912c', 'request_id': 'req-4265e3a9-f8a8-4296-8238-b525b405da44'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.252579 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] The token cache is being invalidate because project af3380a2571c4631a1037faded5f912c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:13.253309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.255277 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.257014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 890/1767] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:13 2022] DELETE /identity/v3/projects/af3380a2571c4631a1037faded5f912c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:13.260886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:13.262687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:13.264304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de0ba6-36c4-4d79-b817-63c75e21af40 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:13.295415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:13.369303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:13.369937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:13.370220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:13.370717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:13.377820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:13.379395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265e3a9-f8a8-4296-8238-b525b405da44 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:13.385653 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:13.386299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:13.386690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:13.387115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] PATH_INFO: `/v3/users/548c3e05cd674246ad287590c2e4193c` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:13.388640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=548c3e05cd674246ad287590c2e4193c)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:13.390582 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:13.391617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:13.394587 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:13.397591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:13.417048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 878/1768] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:13.487386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '548c3e05cd674246ad287590c2e4193c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.497024 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '548c3e05cd674246ad287590c2e4193c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.497883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '548c3e05cd674246ad287590c2e4193c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.498711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '548c3e05cd674246ad287590c2e4193c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.516186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '548c3e05cd674246ad287590c2e4193c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:13.545744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 891/1769] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:13 2022] DELETE /identity/v3/users/548c3e05cd674246ad287590c2e4193c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:14.067713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:14.069580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:14.110118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d16ef-8305-4d2d-918a-a3d6a9f9f207 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:14.147902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:14.155348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:14.275463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac5820-b80f-4185-be1d-5cfbb5d23b80 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:14.278165 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:14.279516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:14.279987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:14.281096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] PATH_INFO: `/v3/projects/6d98f052e0bf46098105ed623fa90c20` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:14.282547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d98f052e0bf46098105ed623fa90c20)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:14.285348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:14.285539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:14.286166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:14.286445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:14.286715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] PATH_INFO: `/v3/projects/1e81592f70ab4e219c04f9934e2eb3f8` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:14.289001 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e81592f70ab4e219c04f9934e2eb3f8)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:14.291463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:14.382914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:14.402338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e81592f70ab4e219c04f9934e2eb3f8', 'request_id': 'req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.415624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e81592f70ab4e219c04f9934e2eb3f8', 'request_id': 'req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.421486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d98f052e0bf46098105ed623fa90c20', 'request_id': 'req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.429173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d98f052e0bf46098105ed623fa90c20', 'request_id': 'req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.439076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e81592f70ab4e219c04f9934e2eb3f8', 'request_id': 'req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.444880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d98f052e0bf46098105ed623fa90c20', 'request_id': 'req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.462878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] The token cache is being invalidate because project 1e81592f70ab4e219c04f9934e2eb3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:14.462878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.463603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-538d77a8-ac6e-4eb0-ae38-fc0aa57454cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.469028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 892/1770] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:14 2022] DELETE /identity/v3/projects/1e81592f70ab4e219c04f9934e2eb3f8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:14.470416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d1ea1-1917-4d3b-b1b6-1ac2d0a4d8f4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:14.470854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d1ea1-1917-4d3b-b1b6-1ac2d0a4d8f4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:14.471142 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d1ea1-1917-4d3b-b1b6-1ac2d0a4d8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:14.475244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] The token cache is being invalidate because project 6d98f052e0bf46098105ed623fa90c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:14.475814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.488720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.495957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 879/1771] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:14 2022] DELETE /identity/v3/projects/6d98f052e0bf46098105ed623fa90c20 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:14.497295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:14.498898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:14.519200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-867d1ea1-1917-4d3b-b1b6-1ac2d0a4d8f4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:14.521120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-867d1ea1-1917-4d3b-b1b6-1ac2d0a4d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:14.584928 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd2bb2-5b73-4902-bfa3-e48a8ecd7f8a admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:14.592845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:14.593504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:14.594007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:14.594293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] PATH_INFO: `/v3/users/2302068cc499439aa1ca6af1a5f4e33f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:14.595876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2302068cc499439aa1ca6af1a5f4e33f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:14.598556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:14.611224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 893/1772] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:14.619138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb1693-d864-4b4c-85f6-2503638a20c3 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:14.619543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb1693-d864-4b4c-85f6-2503638a20c3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:14.619831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb1693-d864-4b4c-85f6-2503638a20c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:14.641896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:14.655948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-10cb1693-d864-4b4c-85f6-2503638a20c3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:14.659143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-10cb1693-d864-4b4c-85f6-2503638a20c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:14.668241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2302068cc499439aa1ca6af1a5f4e33f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.671746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 894/1773] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:42:14.676692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb1693-d864-4b4c-85f6-2503638a20c3 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:14.681563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2302068cc499439aa1ca6af1a5f4e33f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.683132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2302068cc499439aa1ca6af1a5f4e33f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.683645 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2302068cc499439aa1ca6af1a5f4e33f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.689994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb1693-d864-4b4c-85f6-2503638a20c3 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:14.693854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2302068cc499439aa1ca6af1a5f4e33f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:14.696228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:14.697060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:14.697350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:14.697626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:14.698288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:42:14.698654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:14.700862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:14.720407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 895/1774] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:42:14.721480 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:14.724439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 880/1775] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:14 2022] DELETE /identity/v3/users/2302068cc499439aa1ca6af1a5f4e33f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:14.725851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:14.741919 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785409c-2ce8-4516-b659-924b51525680 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:14.849814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f194944-c926-4b51-88b1-c2f87b95957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:14.850659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f194944-c926-4b51-88b1-c2f87b95957f admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:14.851050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f194944-c926-4b51-88b1-c2f87b95957f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:14.851408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f194944-c926-4b51-88b1-c2f87b95957f admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:14.852415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f194944-c926-4b51-88b1-c2f87b95957f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:14.855202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f194944-c926-4b51-88b1-c2f87b95957f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:14.923680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 896/1776] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:14 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:14.924553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:14.939723 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024cb96-79fc-42b5-8c8d-566589d5517e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.029704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecc4b1-0768-4750-8a40-6f87aaf92acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.030713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecc4b1-0768-4750-8a40-6f87aaf92acc admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.031586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecc4b1-0768-4750-8a40-6f87aaf92acc admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.032175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecc4b1-0768-4750-8a40-6f87aaf92acc admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.033962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecc4b1-0768-4750-8a40-6f87aaf92acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.038755 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecc4b1-0768-4750-8a40-6f87aaf92acc admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.083691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 881/1777] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:14 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:15.086546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f194944-c926-4b51-88b1-c2f87b95957f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.118653 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f194944-c926-4b51-88b1-c2f87b95957f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.124214 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecc4b1-0768-4750-8a40-6f87aaf92acc admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.126089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a9ecc4b1-0768-4750-8a40-6f87aaf92acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:15.140194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90831e-c758-4f45-88ef-337ae09e8101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.147652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90831e-c758-4f45-88ef-337ae09e8101 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.148428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90831e-c758-4f45-88ef-337ae09e8101 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.149975 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90831e-c758-4f45-88ef-337ae09e8101 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.150554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90831e-c758-4f45-88ef-337ae09e8101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.150660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecc4b1-0768-4750-8a40-6f87aaf92acc admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.154661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90831e-c758-4f45-88ef-337ae09e8101 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.164845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.170355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.170865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.171763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] PATH_INFO: `/v3/projects/ea451ea7820a45f99b254dbc136d1d39` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.173159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea451ea7820a45f99b254dbc136d1d39)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.175245 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.184273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 897/1778] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:15.188309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90831e-c758-4f45-88ef-337ae09e8101 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.206785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90831e-c758-4f45-88ef-337ae09e8101 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.212487 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee630f-dfdc-4bb7-b619-a7b6332c622e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.213198 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee630f-dfdc-4bb7-b619-a7b6332c622e admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.213543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee630f-dfdc-4bb7-b619-a7b6332c622e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.214347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee630f-dfdc-4bb7-b619-a7b6332c622e admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.215348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee630f-dfdc-4bb7-b619-a7b6332c622e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.217431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee630f-dfdc-4bb7-b619-a7b6332c622e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.229541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 898/1779] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:15.234182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee630f-dfdc-4bb7-b619-a7b6332c622e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.246475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea451ea7820a45f99b254dbc136d1d39', 'request_id': 'req-37c1184e-e26b-420a-b0aa-72186df19042'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:15.248280 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee630f-dfdc-4bb7-b619-a7b6332c622e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.252776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea451ea7820a45f99b254dbc136d1d39', 'request_id': 'req-37c1184e-e26b-420a-b0aa-72186df19042'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:15.265229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:15.268387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea451ea7820a45f99b254dbc136d1d39', 'request_id': 'req-37c1184e-e26b-420a-b0aa-72186df19042'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:15.313927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] The token cache is being invalidate because project ea451ea7820a45f99b254dbc136d1d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:15.314499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:15.315054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-37c1184e-e26b-420a-b0aa-72186df19042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:15.316985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 882/1780] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:15 2022] DELETE /identity/v3/projects/ea451ea7820a45f99b254dbc136d1d39 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:15.326628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5c88-9ff0-478e-9375-6f46139f12e3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.326903 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5c88-9ff0-478e-9375-6f46139f12e3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.327232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5c88-9ff0-478e-9375-6f46139f12e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.361359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a193b6-8c81-4a0c-ac94-c9e95577a3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.362377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a193b6-8c81-4a0c-ac94-c9e95577a3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.362770 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a193b6-8c81-4a0c-ac94-c9e95577a3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.363155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a193b6-8c81-4a0c-ac94-c9e95577a3e6 admin admin] PATH_INFO: `/v3/projects/d5614088125349f1975cc55b25dd9dbc/users/ac5921418d40446c80892b04a3eee12d/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.372763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-7e5a5c88-9ff0-478e-9375-6f46139f12e3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:15.377433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7e5a5c88-9ff0-478e-9375-6f46139f12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:15.386379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 883/1781] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:15.394089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a193b6-8c81-4a0c-ac94-c9e95577a3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5614088125349f1975cc55b25dd9dbc, user_id=ac5921418d40446c80892b04a3eee12d, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.394267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a685117-4fa8-4da3-8a52-8d25304513e8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.394516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a685117-4fa8-4da3-8a52-8d25304513e8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.394794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a685117-4fa8-4da3-8a52-8d25304513e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.398246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a193b6-8c81-4a0c-ac94-c9e95577a3e6 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.419518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-4a685117-4fa8-4da3-8a52-8d25304513e8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:15.420829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4a685117-4fa8-4da3-8a52-8d25304513e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:15.424566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 899/1782] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:15 2022] PUT /identity/v3/projects/d5614088125349f1975cc55b25dd9dbc/users/ac5921418d40446c80892b04a3eee12d/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:15.426979 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:15.428568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b47ec-a151-46fa-bfc5-00cefd75aa05 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.430832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b47ec-a151-46fa-bfc5-00cefd75aa05 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.432289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b47ec-a151-46fa-bfc5-00cefd75aa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.465160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-748b47ec-a151-46fa-bfc5-00cefd75aa05 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:15.469420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-748b47ec-a151-46fa-bfc5-00cefd75aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:15.578583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 884/1783] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:42:15.581183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a685117-4fa8-4da3-8a52-8d25304513e8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.600578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a685117-4fa8-4da3-8a52-8d25304513e8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.613881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ddbe0-60eb-4f8c-b0e9-20651319cbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.614510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ddbe0-60eb-4f8c-b0e9-20651319cbbe admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.614792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ddbe0-60eb-4f8c-b0e9-20651319cbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.615237 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ddbe0-60eb-4f8c-b0e9-20651319cbbe admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.616219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ddbe0-60eb-4f8c-b0e9-20651319cbbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:42:15.616322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 900/1784] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:15.616511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ddbe0-60eb-4f8c-b0e9-20651319cbbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.618155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ddbe0-60eb-4f8c-b0e9-20651319cbbe admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.626278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b47ec-a151-46fa-bfc5-00cefd75aa05 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.628245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-748b47ec-a151-46fa-bfc5-00cefd75aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:15.677424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 885/1785] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:42:15.679579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ddbe0-60eb-4f8c-b0e9-20651319cbbe admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.705998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ddbe0-60eb-4f8c-b0e9-20651319cbbe admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.720706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd734243-b8fb-4fb5-bfe2-163fd310d941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.721874 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd734243-b8fb-4fb5-bfe2-163fd310d941 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.722165 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd734243-b8fb-4fb5-bfe2-163fd310d941 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.722436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd734243-b8fb-4fb5-bfe2-163fd310d941 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.723079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd734243-b8fb-4fb5-bfe2-163fd310d941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.725039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd734243-b8fb-4fb5-bfe2-163fd310d941 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.742740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b47ec-a151-46fa-bfc5-00cefd75aa05 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.747082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484d254-7d2b-4392-98e5-caa8f2c386f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.747546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484d254-7d2b-4392-98e5-caa8f2c386f8 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.747833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484d254-7d2b-4392-98e5-caa8f2c386f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.748614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484d254-7d2b-4392-98e5-caa8f2c386f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.758812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484d254-7d2b-4392-98e5-caa8f2c386f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.762774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484d254-7d2b-4392-98e5-caa8f2c386f8 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.775905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 901/1786] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:42:15.781432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 886/1787] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:15 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:15.785688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484d254-7d2b-4392-98e5-caa8f2c386f8 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.795998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484d254-7d2b-4392-98e5-caa8f2c386f8 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.860247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:15.878565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c98c2-43b8-4f5c-8f3f-c69e66f88917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.879292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c98c2-43b8-4f5c-8f3f-c69e66f88917 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.879557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c98c2-43b8-4f5c-8f3f-c69e66f88917 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.879833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c98c2-43b8-4f5c-8f3f-c69e66f88917 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.880502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c98c2-43b8-4f5c-8f3f-c69e66f88917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.882406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c98c2-43b8-4f5c-8f3f-c69e66f88917 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.914424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 902/1788] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:15 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:15.919591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd734243-b8fb-4fb5-bfe2-163fd310d941 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.932800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd734243-b8fb-4fb5-bfe2-163fd310d941 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.941670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cad6cc-2974-4845-b7ef-08f585023873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.942516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cad6cc-2974-4845-b7ef-08f585023873 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.942869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cad6cc-2974-4845-b7ef-08f585023873 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.943219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cad6cc-2974-4845-b7ef-08f585023873 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.943908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cad6cc-2974-4845-b7ef-08f585023873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.945758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cad6cc-2974-4845-b7ef-08f585023873 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:15.960423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 887/1789] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:15.963326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c98c2-43b8-4f5c-8f3f-c69e66f88917 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:15.979915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c98c2-43b8-4f5c-8f3f-c69e66f88917 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:15.985512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61596e38-cd86-4670-bc24-714b82d42741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:15.986707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61596e38-cd86-4670-bc24-714b82d42741 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:15.987021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61596e38-cd86-4670-bc24-714b82d42741 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:15.987306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61596e38-cd86-4670-bc24-714b82d42741 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:15.987960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61596e38-cd86-4670-bc24-714b82d42741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:15.989692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61596e38-cd86-4670-bc24-714b82d42741 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:16.004877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 903/1790] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:16.006464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cad6cc-2974-4845-b7ef-08f585023873 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:16.019582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cad6cc-2974-4845-b7ef-08f585023873 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:16.024721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cafa3-caaf-4d20-80ee-206d6124a65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:16.025389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cafa3-caaf-4d20-80ee-206d6124a65e admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.025839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cafa3-caaf-4d20-80ee-206d6124a65e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.026126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cafa3-caaf-4d20-80ee-206d6124a65e admin admin] PATH_INFO: `/v3/projects/9acaf2280fa64285ab2035a75de3ee35/users/ba7e51fa8ef343488f63e674da7fc067/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.047502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cafa3-caaf-4d20-80ee-206d6124a65e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9acaf2280fa64285ab2035a75de3ee35, user_id=ba7e51fa8ef343488f63e674da7fc067, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:16.050431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cafa3-caaf-4d20-80ee-206d6124a65e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:16.076190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 888/1791] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:16 2022] PUT /identity/v3/projects/9acaf2280fa64285ab2035a75de3ee35/users/ba7e51fa8ef343488f63e674da7fc067/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:16.078728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c176b-13a1-4675-8444-4197b9d64a02 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.079353 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c176b-13a1-4675-8444-4197b9d64a02 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.079901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c176b-13a1-4675-8444-4197b9d64a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.118982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-fe9c176b-13a1-4675-8444-4197b9d64a02 None None] MFA Rules not processed for user `ba7e51fa8ef343488f63e674da7fc067`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:16.126145 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fe9c176b-13a1-4675-8444-4197b9d64a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:16.220385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 904/1792] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:16.224270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cafa3-caaf-4d20-80ee-206d6124a65e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:16.305790 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cafa3-caaf-4d20-80ee-206d6124a65e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:16.311055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0396da-51f3-44ad-b731-5776042159b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:16.311668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0396da-51f3-44ad-b731-5776042159b1 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.311951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0396da-51f3-44ad-b731-5776042159b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.312247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0396da-51f3-44ad-b731-5776042159b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.313032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0396da-51f3-44ad-b731-5776042159b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:16.314812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0396da-51f3-44ad-b731-5776042159b1 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:16.360356 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 889/1793] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:16 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:16.364211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c176b-13a1-4675-8444-4197b9d64a02 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:16.376849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c176b-13a1-4675-8444-4197b9d64a02 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:16.447726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed14c69-0ba9-4f29-8540-c71517809bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:16.448477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed14c69-0ba9-4f29-8540-c71517809bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.449675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed14c69-0ba9-4f29-8540-c71517809bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.450085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed14c69-0ba9-4f29-8540-c71517809bc0 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.450766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed14c69-0ba9-4f29-8540-c71517809bc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:16.452751 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed14c69-0ba9-4f29-8540-c71517809bc0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:16.478443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 905/1794] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:16 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:16.479694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0396da-51f3-44ad-b731-5776042159b1 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:16.491819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0396da-51f3-44ad-b731-5776042159b1 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:16.497045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09740485-5507-4a38-887d-c259a82152f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:16.497656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09740485-5507-4a38-887d-c259a82152f5 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.497932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09740485-5507-4a38-887d-c259a82152f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.498204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09740485-5507-4a38-887d-c259a82152f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.498847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09740485-5507-4a38-887d-c259a82152f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:16.500520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09740485-5507-4a38-887d-c259a82152f5 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:16.511140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 890/1795] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:16.516403 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed14c69-0ba9-4f29-8540-c71517809bc0 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:16.527608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed14c69-0ba9-4f29-8540-c71517809bc0 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:16.532327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cac13-d6f4-493d-aa65-d3a63e1a694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:16.532936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cac13-d6f4-493d-aa65-d3a63e1a694d admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.533210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cac13-d6f4-493d-aa65-d3a63e1a694d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.533481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cac13-d6f4-493d-aa65-d3a63e1a694d admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.534114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cac13-d6f4-493d-aa65-d3a63e1a694d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:16.535726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cac13-d6f4-493d-aa65-d3a63e1a694d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:16.548383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 906/1796] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:16.550800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09740485-5507-4a38-887d-c259a82152f5 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:16.565261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09740485-5507-4a38-887d-c259a82152f5 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:16.570831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e26ae-5748-4016-98c5-84adfa9c115d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:16.571516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e26ae-5748-4016-98c5-84adfa9c115d admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.571917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e26ae-5748-4016-98c5-84adfa9c115d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.572201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e26ae-5748-4016-98c5-84adfa9c115d admin admin] PATH_INFO: `/v3/projects/54f8f2bd3ff0413b94f3426a8abd1721/users/4f0d83e970df4ade8dcec81d7357a03a/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.588674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:16.592966 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e26ae-5748-4016-98c5-84adfa9c115d admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f8f2bd3ff0413b94f3426a8abd1721, user_id=4f0d83e970df4ade8dcec81d7357a03a, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:16.597340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e26ae-5748-4016-98c5-84adfa9c115d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:16.641668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118df109-8103-4bc7-a7f7-479c09ba3f38 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.641953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118df109-8103-4bc7-a7f7-479c09ba3f38 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.642218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118df109-8103-4bc7-a7f7-479c09ba3f38 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.643888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 907/1797] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:42:16.663330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 891/1798] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:16 2022] PUT /identity/v3/projects/54f8f2bd3ff0413b94f3426a8abd1721/users/4f0d83e970df4ade8dcec81d7357a03a/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:16.674224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022316-4eb1-4232-a1bf-c1bc64b1ddbc None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.674516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022316-4eb1-4232-a1bf-c1bc64b1ddbc None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.675673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022316-4eb1-4232-a1bf-c1bc64b1ddbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.680579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecef9cb-ca6a-49a9-8df3-cbfb8eceb2bc None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.681504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecef9cb-ca6a-49a9-8df3-cbfb8eceb2bc None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.682669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecef9cb-ca6a-49a9-8df3-cbfb8eceb2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.728175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6ecef9cb-ca6a-49a9-8df3-cbfb8eceb2bc None None] MFA Rules not processed for user `fc18a78229064e8d8ea4321c3c23c4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:16.730536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6ecef9cb-ca6a-49a9-8df3-cbfb8eceb2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:16.751236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-18022316-4eb1-4232-a1bf-c1bc64b1ddbc None None] MFA Rules not processed for user `4f0d83e970df4ade8dcec81d7357a03a`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:16.752659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-18022316-4eb1-4232-a1bf-c1bc64b1ddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:16.805888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:16.837259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 892/1799] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:42:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:16.843467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecef9cb-ca6a-49a9-8df3-cbfb8eceb2bc None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:16.845028 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6ecef9cb-ca6a-49a9-8df3-cbfb8eceb2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:16.910178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 908/1800] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:16.912644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f204-b9bf-4181-8880-d2ed3e295c27 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.912644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f204-b9bf-4181-8880-d2ed3e295c27 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.912644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f204-b9bf-4181-8880-d2ed3e295c27 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.916543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 909/1801] 10.222.0.13 () {54 vars in 961 bytes} [Thu Oct 20 14:42:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:42:16.918038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45f204-b9bf-4181-8880-d2ed3e295c27 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:16.955807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecef9cb-ca6a-49a9-8df3-cbfb8eceb2bc None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:16.962840 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:16.963948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:16.965254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:16.966193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] PATH_INFO: `/v3/users/812daba513084a55badecfcde843a6ff` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:16.967637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=812daba513084a55badecfcde843a6ff)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:16.970211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:17.038396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '812daba513084a55badecfcde843a6ff'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.048655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '812daba513084a55badecfcde843a6ff'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.051743 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '812daba513084a55badecfcde843a6ff'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.055632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '812daba513084a55badecfcde843a6ff'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.064731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45f204-b9bf-4181-8880-d2ed3e295c27 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:17.073080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5123de1f-7db2-4c3a-9b52-e57f013f8336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '812daba513084a55badecfcde843a6ff'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.081347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd89e6-ca10-42c3-80b3-b6c34324b794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:17.082335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd89e6-ca10-42c3-80b3-b6c34324b794 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.082782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd89e6-ca10-42c3-80b3-b6c34324b794 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.083218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd89e6-ca10-42c3-80b3-b6c34324b794 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.084153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd89e6-ca10-42c3-80b3-b6c34324b794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:17.086663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd89e6-ca10-42c3-80b3-b6c34324b794 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:17.115973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 893/1802] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:16 2022] DELETE /identity/v3/users/812daba513084a55badecfcde843a6ff => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:17.117371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b155a-66fd-40ae-8794-3020117ecf0d None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.117659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b155a-66fd-40ae-8794-3020117ecf0d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.117943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b155a-66fd-40ae-8794-3020117ecf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.129715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 910/1803] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:16 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:17.134498 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd89e6-ca10-42c3-80b3-b6c34324b794 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:17.135493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:17.140007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c7cd89e6-ca10-42c3-80b3-b6c34324b794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:17.157819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-4b4b155a-66fd-40ae-8794-3020117ecf0d None None] MFA Rules not processed for user `fc18a78229064e8d8ea4321c3c23c4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:17.161321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4b4b155a-66fd-40ae-8794-3020117ecf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:17.283038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd89e6-ca10-42c3-80b3-b6c34324b794 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:17.302604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d477e-c425-4a1a-8b10-a7643fcaa9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:17.303311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d477e-c425-4a1a-8b10-a7643fcaa9ec admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.303522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d477e-c425-4a1a-8b10-a7643fcaa9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.303794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d477e-c425-4a1a-8b10-a7643fcaa9ec admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.304485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d477e-c425-4a1a-8b10-a7643fcaa9ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:17.306409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d477e-c425-4a1a-8b10-a7643fcaa9ec admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:17.315258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 894/1804] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Oct 20 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:17.325305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b155a-66fd-40ae-8794-3020117ecf0d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:17.344404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b155a-66fd-40ae-8794-3020117ecf0d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:17.357708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 911/1805] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:17 2022] POST /identity/v3/users => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:17.362221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d477e-c425-4a1a-8b10-a7643fcaa9ec admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:17.363102 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63f910-10e7-4f24-8287-abbcde63dc99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc18a78229064e8d8ea4321c3c23c4ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:17.363977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63f910-10e7-4f24-8287-abbcde63dc99 service placement] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.364490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63f910-10e7-4f24-8287-abbcde63dc99 service placement] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.364970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63f910-10e7-4f24-8287-abbcde63dc99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.370470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3e63f910-10e7-4f24-8287-abbcde63dc99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:17.389879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d477e-c425-4a1a-8b10-a7643fcaa9ec admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:17.396614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63f910-10e7-4f24-8287-abbcde63dc99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:17.399171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e93a26-7ed6-45a8-8c9f-2f04c0b3220a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:17.399743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e93a26-7ed6-45a8-8c9f-2f04c0b3220a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.400031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e93a26-7ed6-45a8-8c9f-2f04c0b3220a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.400307 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e93a26-7ed6-45a8-8c9f-2f04c0b3220a admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.400989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e93a26-7ed6-45a8-8c9f-2f04c0b3220a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:17.402907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63f910-10e7-4f24-8287-abbcde63dc99 service placement] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:17.403178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e93a26-7ed6-45a8-8c9f-2f04c0b3220a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:17.434648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 895/1806] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:42:17.458914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 912/1807] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:17.460565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63f910-10e7-4f24-8287-abbcde63dc99 service placement] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:17.475960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63f910-10e7-4f24-8287-abbcde63dc99 service placement] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:17.489907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02d285-8955-47cb-8e5b-7ea15c7deb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:17.490586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02d285-8955-47cb-8e5b-7ea15c7deb08 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.490879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02d285-8955-47cb-8e5b-7ea15c7deb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.491157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02d285-8955-47cb-8e5b-7ea15c7deb08 admin admin] PATH_INFO: `/v3/projects/18a0836c389640a4bfeefaad3d93bc8a/users/7ead62b21c6b40f3b378ea43848267c2/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.501561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e93a26-7ed6-45a8-8c9f-2f04c0b3220a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:17.503523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-87e93a26-7ed6-45a8-8c9f-2f04c0b3220a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:17.547087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e93a26-7ed6-45a8-8c9f-2f04c0b3220a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:17.552419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02d285-8955-47cb-8e5b-7ea15c7deb08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a0836c389640a4bfeefaad3d93bc8a, user_id=7ead62b21c6b40f3b378ea43848267c2, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:17.557505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02d285-8955-47cb-8e5b-7ea15c7deb08 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:17.562375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:17.563552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.564631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.565713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] PATH_INFO: `/v3/projects/201aed13c0e342f5b30286ce3f82eb82` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.567800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=201aed13c0e342f5b30286ce3f82eb82)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:17.569935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:17.588663 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 896/1808] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:17 2022] PUT /identity/v3/projects/18a0836c389640a4bfeefaad3d93bc8a/users/7ead62b21c6b40f3b378ea43848267c2/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:17.593131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02d285-8955-47cb-8e5b-7ea15c7deb08 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:17.631784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:17.696234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201aed13c0e342f5b30286ce3f82eb82', 'request_id': 'req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.704523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201aed13c0e342f5b30286ce3f82eb82', 'request_id': 'req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.716778 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201aed13c0e342f5b30286ce3f82eb82', 'request_id': 'req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.734512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] The token cache is being invalidate because project 201aed13c0e342f5b30286ce3f82eb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:17.735069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.736764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-7e1fdf20-02dd-4c89-a4b5-8200ac4b6f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:17.739069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 913/1809] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:17 2022] DELETE /identity/v3/projects/201aed13c0e342f5b30286ce3f82eb82 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:17.741883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02d285-8955-47cb-8e5b-7ea15c7deb08 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:17.744463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fb504-c787-47d8-bd8b-01f3987d7946 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.744845 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fb504-c787-47d8-bd8b-01f3987d7946 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.745171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fb504-c787-47d8-bd8b-01f3987d7946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.778128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-130fb504-c787-47d8-bd8b-01f3987d7946 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:17.779826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-130fb504-c787-47d8-bd8b-01f3987d7946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:17.885872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4669349-7ec4-4fe4-a741-d26077b0375b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:17.887512 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4669349-7ec4-4fe4-a741-d26077b0375b admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.887939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4669349-7ec4-4fe4-a741-d26077b0375b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.888292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4669349-7ec4-4fe4-a741-d26077b0375b admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.889086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4669349-7ec4-4fe4-a741-d26077b0375b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:17.891439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4669349-7ec4-4fe4-a741-d26077b0375b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:17.903387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 897/1810] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:17.910789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4669349-7ec4-4fe4-a741-d26077b0375b admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:17.915113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c4669349-7ec4-4fe4-a741-d26077b0375b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:17.915554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 914/1811] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:17.916831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceb391-0438-4c9e-b8f4-a55b0a5863b0 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.917115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceb391-0438-4c9e-b8f4-a55b0a5863b0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.917975 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceb391-0438-4c9e-b8f4-a55b0a5863b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.929517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4669349-7ec4-4fe4-a741-d26077b0375b admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:17.935594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734935b-383e-4603-bdca-80909b7a9828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:17.936527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734935b-383e-4603-bdca-80909b7a9828 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.936817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734935b-383e-4603-bdca-80909b7a9828 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.937096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734935b-383e-4603-bdca-80909b7a9828 admin admin] PATH_INFO: `/v3/projects/18a0836c389640a4bfeefaad3d93bc8a/users/7ead62b21c6b40f3b378ea43848267c2/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.939199 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734935b-383e-4603-bdca-80909b7a9828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a0836c389640a4bfeefaad3d93bc8a, user_id=7ead62b21c6b40f3b378ea43848267c2, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:17.942339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734935b-383e-4603-bdca-80909b7a9828 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:17.942468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d1ceb391-0438-4c9e-b8f4-a55b0a5863b0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:17.943753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d1ceb391-0438-4c9e-b8f4-a55b0a5863b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:17.946556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:17.955343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 915/1812] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:17.958833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 898/1813] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:17 2022] PUT /identity/v3/projects/18a0836c389640a4bfeefaad3d93bc8a/users/7ead62b21c6b40f3b378ea43848267c2/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:17.961213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ceb391-0438-4c9e-b8f4-a55b0a5863b0 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:17.973007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d66918-6249-4999-8594-d6207620bcde None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:17.973339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d66918-6249-4999-8594-d6207620bcde None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:17.973996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d66918-6249-4999-8594-d6207620bcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:17.976637 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ceb391-0438-4c9e-b8f4-a55b0a5863b0 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:18.018658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-f1d66918-6249-4999-8594-d6207620bcde None None] MFA Rules not processed for user `7ead62b21c6b40f3b378ea43848267c2`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:18.020738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f1d66918-6249-4999-8594-d6207620bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:18.055352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f427115-aed3-49f4-8f0d-cc45ccb910f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:18.056071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f427115-aed3-49f4-8f0d-cc45ccb910f2 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:18.056460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f427115-aed3-49f4-8f0d-cc45ccb910f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:18.056813 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f427115-aed3-49f4-8f0d-cc45ccb910f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:18.057546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f427115-aed3-49f4-8f0d-cc45ccb910f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:42:18.059098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f427115-aed3-49f4-8f0d-cc45ccb910f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:18.061590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f427115-aed3-49f4-8f0d-cc45ccb910f2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:18.076666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 916/1814] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:42:18.080804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f427115-aed3-49f4-8f0d-cc45ccb910f2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:18.096873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f427115-aed3-49f4-8f0d-cc45ccb910f2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:18.104391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce8c4d-ba78-48b9-83ac-f04628cf15ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:18.106363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce8c4d-ba78-48b9-83ac-f04628cf15ec admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:18.107292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce8c4d-ba78-48b9-83ac-f04628cf15ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:18.107618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce8c4d-ba78-48b9-83ac-f04628cf15ec admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:18.108310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce8c4d-ba78-48b9-83ac-f04628cf15ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:18.110640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce8c4d-ba78-48b9-83ac-f04628cf15ec admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:18.148533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 899/1815] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:18.161518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d66918-6249-4999-8594-d6207620bcde None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:18.166204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f1d66918-6249-4999-8594-d6207620bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:18.174985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 917/1816] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:18 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:42:18.179452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce8c4d-ba78-48b9-83ac-f04628cf15ec admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:18.196554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:18.270117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d66918-6249-4999-8594-d6207620bcde None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:18.342906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce8c4d-ba78-48b9-83ac-f04628cf15ec admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:18.360432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c7e5b-1d8f-4207-9ff9-02b6ba61ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:18.362644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c7e5b-1d8f-4207-9ff9-02b6ba61ddc5 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:18.362787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c7e5b-1d8f-4207-9ff9-02b6ba61ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:18.363389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c7e5b-1d8f-4207-9ff9-02b6ba61ddc5 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:18.363963 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c7e5b-1d8f-4207-9ff9-02b6ba61ddc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:18.366147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c7e5b-1d8f-4207-9ff9-02b6ba61ddc5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:18.370266 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b66528-75b7-450d-9ee9-d0b3ef09e51b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:18.370943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66528-75b7-450d-9ee9-d0b3ef09e51b service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:18.371496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66528-75b7-450d-9ee9-d0b3ef09e51b service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:18.371496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66528-75b7-450d-9ee9-d0b3ef09e51b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:18.373271 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a6b66528-75b7-450d-9ee9-d0b3ef09e51b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:18.396649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 918/1817] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:18 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:18.400998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c7e5b-1d8f-4207-9ff9-02b6ba61ddc5 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:18.413393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c7e5b-1d8f-4207-9ff9-02b6ba61ddc5 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:18.419562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b557f-e3a8-4187-8634-4f6503713f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:18.420132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b557f-e3a8-4187-8634-4f6503713f10 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:18.420460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b557f-e3a8-4187-8634-4f6503713f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:18.420734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b557f-e3a8-4187-8634-4f6503713f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:18.421588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b557f-e3a8-4187-8634-4f6503713f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:18.423121 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b557f-e3a8-4187-8634-4f6503713f10 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:18.447458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:18.457651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 919/1818] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:18.459021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b557f-e3a8-4187-8634-4f6503713f10 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:18.461448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b66528-75b7-450d-9ee9-d0b3ef09e51b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:18.463094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b66528-75b7-450d-9ee9-d0b3ef09e51b service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:18.486631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 900/1819] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:18.497169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b557f-e3a8-4187-8634-4f6503713f10 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:18.504434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf656b56-e70d-47c2-9092-0f4c30b41531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:18.505020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf656b56-e70d-47c2-9092-0f4c30b41531 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:18.507138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf656b56-e70d-47c2-9092-0f4c30b41531 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:18.507138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf656b56-e70d-47c2-9092-0f4c30b41531 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:18.507138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf656b56-e70d-47c2-9092-0f4c30b41531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:18.509470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf656b56-e70d-47c2-9092-0f4c30b41531 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:18.529890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 920/1820] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:18.534422 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b66528-75b7-450d-9ee9-d0b3ef09e51b service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:18.546871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b66528-75b7-450d-9ee9-d0b3ef09e51b service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:18.554715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd453fc-4c5a-4357-8293-ae84a73d7a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:18.555498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd453fc-4c5a-4357-8293-ae84a73d7a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:18.555907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd453fc-4c5a-4357-8293-ae84a73d7a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:18.556298 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd453fc-4c5a-4357-8293-ae84a73d7a1c admin admin] PATH_INFO: `/v3/projects/7688b0fbf84b4f4f9c21cb4112ee53b5/users/ae94a3ece5f644c6893589a1df84d1c6/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:18.576337 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd453fc-4c5a-4357-8293-ae84a73d7a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7688b0fbf84b4f4f9c21cb4112ee53b5, user_id=ae94a3ece5f644c6893589a1df84d1c6, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:18.580564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd453fc-4c5a-4357-8293-ae84a73d7a1c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:18.615462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 901/1821] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:18 2022] PUT /identity/v3/projects/7688b0fbf84b4f4f9c21cb4112ee53b5/users/ae94a3ece5f644c6893589a1df84d1c6/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:18.617687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810b305-0db3-4937-accc-d013abe98829 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:18.617923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810b305-0db3-4937-accc-d013abe98829 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:18.618347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810b305-0db3-4937-accc-d013abe98829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:18.640743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c810b305-0db3-4937-accc-d013abe98829 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:18.641979 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c810b305-0db3-4937-accc-d013abe98829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:18.744049 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 921/1822] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:18.756041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd453fc-4c5a-4357-8293-ae84a73d7a1c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:18.758424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-afd453fc-4c5a-4357-8293-ae84a73d7a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:18.851233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd453fc-4c5a-4357-8293-ae84a73d7a1c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:18.857185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32592c4b-f353-4637-a799-c12ab0d90e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:18.857816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32592c4b-f353-4637-a799-c12ab0d90e26 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:18.858099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32592c4b-f353-4637-a799-c12ab0d90e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:18.858376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32592c4b-f353-4637-a799-c12ab0d90e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:18.870574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32592c4b-f353-4637-a799-c12ab0d90e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:18.873195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32592c4b-f353-4637-a799-c12ab0d90e26 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:18.916367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 902/1823] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:42:20.391632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810b305-0db3-4937-accc-d013abe98829 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:20.517958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810b305-0db3-4937-accc-d013abe98829 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:20.523113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b186371-377a-439a-8d8a-f5c5a2ae114d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:20.523936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b186371-377a-439a-8d8a-f5c5a2ae114d service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:20.524392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b186371-377a-439a-8d8a-f5c5a2ae114d service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:20.524815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b186371-377a-439a-8d8a-f5c5a2ae114d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:20.527305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8b186371-377a-439a-8d8a-f5c5a2ae114d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:20.595007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:20.630797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b186371-377a-439a-8d8a-f5c5a2ae114d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:20.632319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b186371-377a-439a-8d8a-f5c5a2ae114d service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:20.675582 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 922/1824] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:21.309778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32592c4b-f353-4637-a799-c12ab0d90e26 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:21.311830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-32592c4b-f353-4637-a799-c12ab0d90e26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:21.333992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32592c4b-f353-4637-a799-c12ab0d90e26 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:21.347873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:21.348614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:21.348879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:21.349161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] PATH_INFO: `/v3/users/c093bc7e38bf494495e917322b44d0a4` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:21.354274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c093bc7e38bf494495e917322b44d0a4)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:21.357210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:21.482686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c093bc7e38bf494495e917322b44d0a4'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:21.494524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c093bc7e38bf494495e917322b44d0a4'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:21.500296 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c093bc7e38bf494495e917322b44d0a4'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:21.502588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c093bc7e38bf494495e917322b44d0a4'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:21.514531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3613493c-b99b-4dd7-8da9-633dca1eb1ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c093bc7e38bf494495e917322b44d0a4'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:21.561642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 903/1825] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:21 2022] DELETE /identity/v3/users/c093bc7e38bf494495e917322b44d0a4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:21.753690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b186371-377a-439a-8d8a-f5c5a2ae114d service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:21.764820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b186371-377a-439a-8d8a-f5c5a2ae114d service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:21.771144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5183a7-c8d9-4bb0-b5b0-91081b0785e3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:21.771144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5183a7-c8d9-4bb0-b5b0-91081b0785e3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:21.771425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5183a7-c8d9-4bb0-b5b0-91081b0785e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:21.811047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5a5183a7-c8d9-4bb0-b5b0-91081b0785e3 None None] MFA Rules not processed for user `ac5921418d40446c80892b04a3eee12d`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:21.812473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5a5183a7-c8d9-4bb0-b5b0-91081b0785e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:21.841621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:21.842633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:21.843172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:21.843631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:21.845933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:21.856894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:21.932023 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 904/1826] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:21.933214 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5183a7-c8d9-4bb0-b5b0-91081b0785e3 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:21.934628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5a5183a7-c8d9-4bb0-b5b0-91081b0785e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:21.945268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:21.948316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:21.965866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 923/1827] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:22.043612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:22.045598 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:22.061994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5183a7-c8d9-4bb0-b5b0-91081b0785e3 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:22.067501 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ce228-6e17-425b-932e-b27d7cb0b9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:22.068156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc171a-9c72-49a3-8554-16afaa22677a service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:22.068279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ce228-6e17-425b-932e-b27d7cb0b9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:22.069538 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ce228-6e17-425b-932e-b27d7cb0b9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:22.069836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ce228-6e17-425b-932e-b27d7cb0b9d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:22.070489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ce228-6e17-425b-932e-b27d7cb0b9d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:22.072718 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ce228-6e17-425b-932e-b27d7cb0b9d9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:22.086536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:22.087203 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:22.087610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:22.087901 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] PATH_INFO: `/v3/projects/8db062fc1ab147a5a513f6f8d776be21` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:22.089308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db062fc1ab147a5a513f6f8d776be21)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:22.091642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:22.123306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 905/1828] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:21 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:22.127318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ce228-6e17-425b-932e-b27d7cb0b9d9 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:22.169020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db062fc1ab147a5a513f6f8d776be21', 'request_id': 'req-2275c86a-5aca-49eb-99db-1cf720ab3cf5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.175258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db062fc1ab147a5a513f6f8d776be21', 'request_id': 'req-2275c86a-5aca-49eb-99db-1cf720ab3cf5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.186343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db062fc1ab147a5a513f6f8d776be21', 'request_id': 'req-2275c86a-5aca-49eb-99db-1cf720ab3cf5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.205805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] The token cache is being invalidate because project 8db062fc1ab147a5a513f6f8d776be21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:22.206100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.207182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.209392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 924/1829] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:22 2022] DELETE /identity/v3/projects/8db062fc1ab147a5a513f6f8d776be21 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:22.213199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:22.216957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:22.235590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:22.267029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ce228-6e17-425b-932e-b27d7cb0b9d9 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:22.397909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275c86a-5aca-49eb-99db-1cf720ab3cf5 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:22.411765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:22.412075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:22.412401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:22.412939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] PATH_INFO: `/v3/users/6e2ec1218d724b4caf1a98e589efbfa0` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:22.416978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2ec1218d724b4caf1a98e589efbfa0)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:22.420439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:22.446912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60117449-8617-4665-a531-269950e79f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:22.447482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60117449-8617-4665-a531-269950e79f15 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:22.447758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60117449-8617-4665-a531-269950e79f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:22.447987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60117449-8617-4665-a531-269950e79f15 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:22.451032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60117449-8617-4665-a531-269950e79f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:22.453023 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60117449-8617-4665-a531-269950e79f15 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:22.469223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:22.499067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 906/1830] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:22 2022] POST /identity/v3/users => generated 296 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:22.511304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60117449-8617-4665-a531-269950e79f15 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:22.513956 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-60117449-8617-4665-a531-269950e79f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:22.527730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60117449-8617-4665-a531-269950e79f15 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:22.533817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e442ec-c173-468c-92ea-34afd75356f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:22.537875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e442ec-c173-468c-92ea-34afd75356f7 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:22.538529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e442ec-c173-468c-92ea-34afd75356f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:22.538960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2ec1218d724b4caf1a98e589efbfa0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.539201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e442ec-c173-468c-92ea-34afd75356f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:22.540151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e442ec-c173-468c-92ea-34afd75356f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:22.542725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e442ec-c173-468c-92ea-34afd75356f7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:22.554666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 907/1831] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:22.560205 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e442ec-c173-468c-92ea-34afd75356f7 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:22.563300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2ec1218d724b4caf1a98e589efbfa0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.564656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2ec1218d724b4caf1a98e589efbfa0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.565887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2ec1218d724b4caf1a98e589efbfa0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.575655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e442ec-c173-468c-92ea-34afd75356f7 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:22.581201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2ec1218d724b4caf1a98e589efbfa0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:22.581623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65acd55-70d1-46bc-92e6-b12ee98de8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:22.582319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65acd55-70d1-46bc-92e6-b12ee98de8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:22.582602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65acd55-70d1-46bc-92e6-b12ee98de8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:22.582878 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65acd55-70d1-46bc-92e6-b12ee98de8c4 admin admin] PATH_INFO: `/v3/projects/2a583792afbd4bf2b016df8fe868fc82/users/cc1a3fb9270a435b819d3a05cf1ed153/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:22.609225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65acd55-70d1-46bc-92e6-b12ee98de8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a583792afbd4bf2b016df8fe868fc82, user_id=cc1a3fb9270a435b819d3a05cf1ed153, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:22.611926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 925/1832] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:22 2022] DELETE /identity/v3/users/6e2ec1218d724b4caf1a98e589efbfa0 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:22.615290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65acd55-70d1-46bc-92e6-b12ee98de8c4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:22.642806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 908/1833] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:22 2022] PUT /identity/v3/projects/2a583792afbd4bf2b016df8fe868fc82/users/cc1a3fb9270a435b819d3a05cf1ed153/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:22.646493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:22.648174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:22.754907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcb252-c5fe-4661-a99d-bd46b601f5a7 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:22.842893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:22.908909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242335b3-5b15-4919-8a19-88a068a0b02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:22.909695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242335b3-5b15-4919-8a19-88a068a0b02d admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:22.910082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242335b3-5b15-4919-8a19-88a068a0b02d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:22.910460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242335b3-5b15-4919-8a19-88a068a0b02d admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:22.911234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242335b3-5b15-4919-8a19-88a068a0b02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:22.913138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242335b3-5b15-4919-8a19-88a068a0b02d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:22.936455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 926/1834] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:42:22.940426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65acd55-70d1-46bc-92e6-b12ee98de8c4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:22.951271 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65acd55-70d1-46bc-92e6-b12ee98de8c4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:22.969860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5232f-8828-4f79-a525-53c12d443812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:22.977872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5232f-8828-4f79-a525-53c12d443812 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:22.980571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5232f-8828-4f79-a525-53c12d443812 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:22.981009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5232f-8828-4f79-a525-53c12d443812 admin admin] PATH_INFO: `/v3/projects/2a583792afbd4bf2b016df8fe868fc82/users/cc1a3fb9270a435b819d3a05cf1ed153/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:22.984129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5232f-8828-4f79-a525-53c12d443812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a583792afbd4bf2b016df8fe868fc82, user_id=cc1a3fb9270a435b819d3a05cf1ed153, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:22.988099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5232f-8828-4f79-a525-53c12d443812 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:23.002321 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 909/1835] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:22 2022] PUT /identity/v3/projects/2a583792afbd4bf2b016df8fe868fc82/users/cc1a3fb9270a435b819d3a05cf1ed153/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:23.003032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242335b3-5b15-4919-8a19-88a068a0b02d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:23.006949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-242335b3-5b15-4919-8a19-88a068a0b02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:23.030601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242335b3-5b15-4919-8a19-88a068a0b02d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:23.161061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:23.162315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:23.162995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:23.163491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] PATH_INFO: `/v3/projects/c6620e37a86642dead8bffe716fc6bac` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:23.165518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6620e37a86642dead8bffe716fc6bac)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:23.168014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:23.271168 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6620e37a86642dead8bffe716fc6bac', 'request_id': 'req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:23.288088 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6620e37a86642dead8bffe716fc6bac', 'request_id': 'req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:23.306978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6620e37a86642dead8bffe716fc6bac', 'request_id': 'req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:23.326326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] The token cache is being invalidate because project c6620e37a86642dead8bffe716fc6bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:23.326631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:23.327334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8f83e703-91ac-42c0-ae46-72e43ce7c3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:23.335082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 927/1836] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:23 2022] DELETE /identity/v3/projects/c6620e37a86642dead8bffe716fc6bac => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:23.337998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ede58b-ce76-47ff-ab9c-850356cf52d8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:23.338367 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ede58b-ce76-47ff-ab9c-850356cf52d8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:23.338642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ede58b-ce76-47ff-ab9c-850356cf52d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:23.379066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-97ede58b-ce76-47ff-ab9c-850356cf52d8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:23.381963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-97ede58b-ce76-47ff-ab9c-850356cf52d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:23.628037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 910/1837] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:23.635766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0f7fd-91f1-4648-88cb-868cd54edf9b None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:23.636814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0f7fd-91f1-4648-88cb-868cd54edf9b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:23.637200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0f7fd-91f1-4648-88cb-868cd54edf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:23.697824 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-66d0f7fd-91f1-4648-88cb-868cd54edf9b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:23.700967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-66d0f7fd-91f1-4648-88cb-868cd54edf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:23.737448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 928/1838] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:23.743557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ede58b-ce76-47ff-ab9c-850356cf52d8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:23.759515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ede58b-ce76-47ff-ab9c-850356cf52d8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:23.786907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdffeba-9d31-4340-823b-695f238097e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:23.787799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdffeba-9d31-4340-823b-695f238097e9 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:23.788241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdffeba-9d31-4340-823b-695f238097e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:23.797340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdffeba-9d31-4340-823b-695f238097e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:23.798639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdffeba-9d31-4340-823b-695f238097e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:42:23.799250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdffeba-9d31-4340-823b-695f238097e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:23.801186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdffeba-9d31-4340-823b-695f238097e9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:23.832290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 911/1839] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:42:23.833338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0f7fd-91f1-4648-88cb-868cd54edf9b None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:23.850268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0f7fd-91f1-4648-88cb-868cd54edf9b None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:23.865721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f5967-4e1b-48d5-9473-2fe4d0732f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:23.867406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f5967-4e1b-48d5-9473-2fe4d0732f17 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:23.867696 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f5967-4e1b-48d5-9473-2fe4d0732f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:23.867970 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f5967-4e1b-48d5-9473-2fe4d0732f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:23.868639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f5967-4e1b-48d5-9473-2fe4d0732f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:23.880729 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f5967-4e1b-48d5-9473-2fe4d0732f17 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:23.988592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 929/1840] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:23 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:23.990452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdffeba-9d31-4340-823b-695f238097e9 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:24.008075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdffeba-9d31-4340-823b-695f238097e9 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:24.092854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbee33-cde5-41e6-b276-77579acb4fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:24.093694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbee33-cde5-41e6-b276-77579acb4fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:24.094044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbee33-cde5-41e6-b276-77579acb4fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:24.094387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbee33-cde5-41e6-b276-77579acb4fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:24.095111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbee33-cde5-41e6-b276-77579acb4fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:24.097217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbee33-cde5-41e6-b276-77579acb4fc5 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:24.133806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 912/1841] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:23 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:24.134986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f5967-4e1b-48d5-9473-2fe4d0732f17 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:24.153772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f5967-4e1b-48d5-9473-2fe4d0732f17 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:24.159695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69cd17-69e5-422f-b8c2-a063f666dc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:24.161443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69cd17-69e5-422f-b8c2-a063f666dc51 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:24.161933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69cd17-69e5-422f-b8c2-a063f666dc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:24.162371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69cd17-69e5-422f-b8c2-a063f666dc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:24.163389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69cd17-69e5-422f-b8c2-a063f666dc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:24.165319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69cd17-69e5-422f-b8c2-a063f666dc51 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:24.189447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 930/1842] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:24.193899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbee33-cde5-41e6-b276-77579acb4fc5 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:24.208450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbee33-cde5-41e6-b276-77579acb4fc5 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:24.216418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9569b65-ef3a-42ed-b66a-3238a1d9b05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:24.217157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569b65-ef3a-42ed-b66a-3238a1d9b05d admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:24.217952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569b65-ef3a-42ed-b66a-3238a1d9b05d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:24.218347 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569b65-ef3a-42ed-b66a-3238a1d9b05d admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:24.219106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9569b65-ef3a-42ed-b66a-3238a1d9b05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:24.224308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9569b65-ef3a-42ed-b66a-3238a1d9b05d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:24.260381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 913/1843] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:24.264576 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69cd17-69e5-422f-b8c2-a063f666dc51 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:24.278738 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69cd17-69e5-422f-b8c2-a063f666dc51 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:24.285154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a9a1-acc6-4c85-9bda-ac122737cb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:24.286177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a9a1-acc6-4c85-9bda-ac122737cb57 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:24.287495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a9a1-acc6-4c85-9bda-ac122737cb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:24.288016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a9a1-acc6-4c85-9bda-ac122737cb57 admin admin] PATH_INFO: `/v3/projects/4f5ac0bff4d445448eb7e7d262d2feaf/users/d1c3e55b6a954f319583fd698171717a/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:24.331303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7a9a1-acc6-4c85-9bda-ac122737cb57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5ac0bff4d445448eb7e7d262d2feaf, user_id=d1c3e55b6a954f319583fd698171717a, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:24.335146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7a9a1-acc6-4c85-9bda-ac122737cb57 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:24.340786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:24.370044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 931/1844] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:24 2022] PUT /identity/v3/projects/4f5ac0bff4d445448eb7e7d262d2feaf/users/d1c3e55b6a954f319583fd698171717a/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:24.382126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc5bf0-1ab7-4152-9337-0704ec1698cf None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:24.382668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc5bf0-1ab7-4152-9337-0704ec1698cf None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:24.383045 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc5bf0-1ab7-4152-9337-0704ec1698cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:24.422344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:24.434358 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-36bc5bf0-1ab7-4152-9337-0704ec1698cf None None] MFA Rules not processed for user `d1c3e55b6a954f319583fd698171717a`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:24.449959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-36bc5bf0-1ab7-4152-9337-0704ec1698cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:24.658155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 914/1845] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:24.670815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a9a1-acc6-4c85-9bda-ac122737cb57 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:24.672931 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fdd7a9a1-acc6-4c85-9bda-ac122737cb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:24.829219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a9a1-acc6-4c85-9bda-ac122737cb57 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:24.837782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98172414-56e0-46ad-998c-f55d525c4530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:24.839660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98172414-56e0-46ad-998c-f55d525c4530 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:24.840943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98172414-56e0-46ad-998c-f55d525c4530 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:24.841444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98172414-56e0-46ad-998c-f55d525c4530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:24.859492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98172414-56e0-46ad-998c-f55d525c4530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:24.861805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98172414-56e0-46ad-998c-f55d525c4530 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:24.898096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 932/1846] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:24.924290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc5bf0-1ab7-4152-9337-0704ec1698cf None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:25.033789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc5bf0-1ab7-4152-9337-0704ec1698cf None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:25.077722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:25.078694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:25.079165 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:25.079688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] PATH_INFO: `/v3/users/d1c3e55b6a954f319583fd698171717a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:25.082108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c3e55b6a954f319583fd698171717a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:25.089469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:25.221450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c3e55b6a954f319583fd698171717a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:25.231650 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c3e55b6a954f319583fd698171717a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:25.232477 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c3e55b6a954f319583fd698171717a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:25.233107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c3e55b6a954f319583fd698171717a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:25.251436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c3e55b6a954f319583fd698171717a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:25.289090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 915/1847] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:24 2022] DELETE /identity/v3/users/d1c3e55b6a954f319583fd698171717a => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:25.293774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294542d-0bed-4c6d-b6df-dc8c23b558d9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:25.294561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294542d-0bed-4c6d-b6df-dc8c23b558d9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:25.294998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294542d-0bed-4c6d-b6df-dc8c23b558d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:25.347708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e294542d-0bed-4c6d-b6df-dc8c23b558d9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:25.352468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e294542d-0bed-4c6d-b6df-dc8c23b558d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:25.501028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 933/1848] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:25.512038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:25.514044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:25.559776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:25.684945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92ddc8-4c94-4457-98a5-c3950de76fb7 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:25.700146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1ded1-3f95-46df-8231-41fa3dd53448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:25.700842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1ded1-3f95-46df-8231-41fa3dd53448 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:25.701135 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1ded1-3f95-46df-8231-41fa3dd53448 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:25.701410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1ded1-3f95-46df-8231-41fa3dd53448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:25.745103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1ded1-3f95-46df-8231-41fa3dd53448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:25.746850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1ded1-3f95-46df-8231-41fa3dd53448 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:25.818517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 916/1849] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:26.987657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294542d-0bed-4c6d-b6df-dc8c23b558d9 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:26.989306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e294542d-0bed-4c6d-b6df-dc8c23b558d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:27.006595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294542d-0bed-4c6d-b6df-dc8c23b558d9 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:27.012142 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:27.012918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:27.013285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:27.013569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] PATH_INFO: `/v3/users/ba7e51fa8ef343488f63e674da7fc067` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:27.016180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba7e51fa8ef343488f63e674da7fc067)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:27.018484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:27.088285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1ded1-3f95-46df-8231-41fa3dd53448 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:27.094556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fad1ded1-3f95-46df-8231-41fa3dd53448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:27.134813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1ded1-3f95-46df-8231-41fa3dd53448 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:27.147482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba7e51fa8ef343488f63e674da7fc067'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.153335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:27.156539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:27.158209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:27.158641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] PATH_INFO: `/v3/projects/4f5ac0bff4d445448eb7e7d262d2feaf` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:27.165879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f5ac0bff4d445448eb7e7d262d2feaf)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:27.170221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:27.183220 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba7e51fa8ef343488f63e674da7fc067'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.184282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba7e51fa8ef343488f63e674da7fc067'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.185172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba7e51fa8ef343488f63e674da7fc067'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.201183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2e3b44cf-46c6-4387-b4cc-da0939d5e66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba7e51fa8ef343488f63e674da7fc067'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.250120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 934/1850] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:26 2022] DELETE /identity/v3/users/ba7e51fa8ef343488f63e674da7fc067 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:27.250120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823404c2-7f7f-4e95-bb76-92bcff9f0801 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:27.250120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823404c2-7f7f-4e95-bb76-92bcff9f0801 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:27.250120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823404c2-7f7f-4e95-bb76-92bcff9f0801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:27.275804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f5ac0bff4d445448eb7e7d262d2feaf', 'request_id': 'req-913f4133-6261-401e-a6fb-c6a1d418f288'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.290368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f5ac0bff4d445448eb7e7d262d2feaf', 'request_id': 'req-913f4133-6261-401e-a6fb-c6a1d418f288'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.297830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:27.305357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f5ac0bff4d445448eb7e7d262d2feaf', 'request_id': 'req-913f4133-6261-401e-a6fb-c6a1d418f288'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.316176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-823404c2-7f7f-4e95-bb76-92bcff9f0801 None None] MFA Rules not processed for user `ae94a3ece5f644c6893589a1df84d1c6`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:27.320206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-823404c2-7f7f-4e95-bb76-92bcff9f0801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:27.330017 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] The token cache is being invalidate because project 4f5ac0bff4d445448eb7e7d262d2feaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:27.330598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.332707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-913f4133-6261-401e-a6fb-c6a1d418f288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:27.335811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 917/1851] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:27 2022] DELETE /identity/v3/projects/4f5ac0bff4d445448eb7e7d262d2feaf => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:27.340309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556fae6-5c81-4040-983f-ea6e08bd9ca3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:27.340654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556fae6-5c81-4040-983f-ea6e08bd9ca3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:27.340938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556fae6-5c81-4040-983f-ea6e08bd9ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:27.416228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-0556fae6-5c81-4040-983f-ea6e08bd9ca3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:27.418981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0556fae6-5c81-4040-983f-ea6e08bd9ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:27.521362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 935/1852] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:27.537206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec977b-f93b-4173-aff8-ef9389429c2f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:27.537621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec977b-f93b-4173-aff8-ef9389429c2f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:27.538009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec977b-f93b-4173-aff8-ef9389429c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:27.576690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 918/1853] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:27.581393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1eec977b-f93b-4173-aff8-ef9389429c2f None None] MFA Rules not processed for user `ae94a3ece5f644c6893589a1df84d1c6`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:27.583144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1eec977b-f93b-4173-aff8-ef9389429c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:27.583463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556fae6-5c81-4040-983f-ea6e08bd9ca3 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:27.589046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0556fae6-5c81-4040-983f-ea6e08bd9ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:27.609521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 936/1854] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:27.619421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec977b-f93b-4173-aff8-ef9389429c2f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:27.622292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1eec977b-f93b-4173-aff8-ef9389429c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:27.695139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:27.704682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556fae6-5c81-4040-983f-ea6e08bd9ca3 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:27.713438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b4800-eaee-495f-ada4-31b6ecdeccad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:27.716076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b4800-eaee-495f-ada4-31b6ecdeccad service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:27.716562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b4800-eaee-495f-ada4-31b6ecdeccad service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:27.717006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b4800-eaee-495f-ada4-31b6ecdeccad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:27.732522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b4800-eaee-495f-ada4-31b6ecdeccad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:27.733963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b4800-eaee-495f-ada4-31b6ecdeccad service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:27.749517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec977b-f93b-4173-aff8-ef9389429c2f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:27.752130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 919/1855] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:27.757689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6a420-ff93-48fb-9631-18768d3d0a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:27.758607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6a420-ff93-48fb-9631-18768d3d0a7a service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:27.758739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6a420-ff93-48fb-9631-18768d3d0a7a service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:27.759031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6a420-ff93-48fb-9631-18768d3d0a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:27.769249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6a420-ff93-48fb-9631-18768d3d0a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:27.770885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6a420-ff93-48fb-9631-18768d3d0a7a service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:27.796975 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 937/1856] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:28.714426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b4800-eaee-495f-ada4-31b6ecdeccad service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:28.716761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6d5b4800-eaee-495f-ada4-31b6ecdeccad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:28.740863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b4800-eaee-495f-ada4-31b6ecdeccad service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:28.756894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:28.757761 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:28.758211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:28.758721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] PATH_INFO: `/v3/projects/9acaf2280fa64285ab2035a75de3ee35` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:28.762035 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9acaf2280fa64285ab2035a75de3ee35)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:28.763990 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:28.879446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9acaf2280fa64285ab2035a75de3ee35', 'request_id': 'req-29170ab6-7e83-4842-a3ca-7278d5e10b8b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:28.894285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9acaf2280fa64285ab2035a75de3ee35', 'request_id': 'req-29170ab6-7e83-4842-a3ca-7278d5e10b8b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:28.913829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9acaf2280fa64285ab2035a75de3ee35', 'request_id': 'req-29170ab6-7e83-4842-a3ca-7278d5e10b8b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:28.930104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] The token cache is being invalidate because project 9acaf2280fa64285ab2035a75de3ee35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:28.930644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:28.934431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-29170ab6-7e83-4842-a3ca-7278d5e10b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:28.949918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6a420-ff93-48fb-9631-18768d3d0a7a service nova] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:28.965967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 920/1857] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:28 2022] DELETE /identity/v3/projects/9acaf2280fa64285ab2035a75de3ee35 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:28.969240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6a420-ff93-48fb-9631-18768d3d0a7a service nova] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:29.145918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:29.147732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:29.148316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:29.148790 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] PATH_INFO: `/v3/users/4f0d83e970df4ade8dcec81d7357a03a` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:29.150748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0d83e970df4ade8dcec81d7357a03a)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:29.155797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:29.220775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:29.305002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0d83e970df4ade8dcec81d7357a03a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:29.331270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0d83e970df4ade8dcec81d7357a03a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:29.331270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0d83e970df4ade8dcec81d7357a03a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:29.331270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0d83e970df4ade8dcec81d7357a03a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:29.344739 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2ded-2e47-4b3f-b6da-9da63a0463ab None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:29.345079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2ded-2e47-4b3f-b6da-9da63a0463ab None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:29.345374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2ded-2e47-4b3f-b6da-9da63a0463ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:29.346163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0d83e970df4ade8dcec81d7357a03a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:29.394584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d59d2ded-2e47-4b3f-b6da-9da63a0463ab None None] MFA Rules not processed for user `ae94a3ece5f644c6893589a1df84d1c6`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:29.398916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d59d2ded-2e47-4b3f-b6da-9da63a0463ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:29.404970 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 938/1858] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:28 2022] DELETE /identity/v3/users/4f0d83e970df4ade8dcec81d7357a03a => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:42:29.529506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 921/1859] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:42:30.665168 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:30.667231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:30.821045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98492fd-e930-4441-8457-111adf2287f0 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:30.828833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:30.830335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:30.831326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:30.831715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] PATH_INFO: `/v3/projects/54f8f2bd3ff0413b94f3426a8abd1721` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:30.834041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f8f2bd3ff0413b94f3426a8abd1721)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:30.836318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:30.918695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f8f2bd3ff0413b94f3426a8abd1721', 'request_id': 'req-4e3b1137-b520-4ab4-9d22-5b57bbf06793'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:30.929742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f8f2bd3ff0413b94f3426a8abd1721', 'request_id': 'req-4e3b1137-b520-4ab4-9d22-5b57bbf06793'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:30.946213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f8f2bd3ff0413b94f3426a8abd1721', 'request_id': 'req-4e3b1137-b520-4ab4-9d22-5b57bbf06793'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:30.966881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] The token cache is being invalidate because project 54f8f2bd3ff0413b94f3426a8abd1721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:30.968700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:30.971325 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4e3b1137-b520-4ab4-9d22-5b57bbf06793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:30.973480 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 939/1860] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:30 2022] DELETE /identity/v3/projects/54f8f2bd3ff0413b94f3426a8abd1721 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:30.977378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2ded-2e47-4b3f-b6da-9da63a0463ab None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:30.997030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2ded-2e47-4b3f-b6da-9da63a0463ab None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:31.141395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:31.142183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:31.144219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:31.145252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] PATH_INFO: `/v3/users/7ead62b21c6b40f3b378ea43848267c2` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:31.151913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ead62b21c6b40f3b378ea43848267c2)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:31.156031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:31.218054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:31.305626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ead62b21c6b40f3b378ea43848267c2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:31.323978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ead62b21c6b40f3b378ea43848267c2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:31.325206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ead62b21c6b40f3b378ea43848267c2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:31.325976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ead62b21c6b40f3b378ea43848267c2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:31.352011 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ead62b21c6b40f3b378ea43848267c2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:31.405339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 922/1861] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:30 2022] DELETE /identity/v3/users/7ead62b21c6b40f3b378ea43848267c2 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:31.841591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1372-ae9e-40ed-baf1-078b2fb390b0 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:31.842745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1372-ae9e-40ed-baf1-078b2fb390b0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:31.843589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1372-ae9e-40ed-baf1-078b2fb390b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:31.944035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e08d1372-ae9e-40ed-baf1-078b2fb390b0 None None] MFA Rules not processed for user `cc1a3fb9270a435b819d3a05cf1ed153`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:31.946577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e08d1372-ae9e-40ed-baf1-078b2fb390b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:31.996926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:32.127841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 940/1862] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:42:32.142159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:32.144331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:32.320143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24405af-cac9-479d-bfa8-7d476d3e55d0 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:32.334472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:32.335168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:32.335371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:32.335644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:32.343716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:32.528741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:32.530383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:32.556047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 923/1863] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:32.604947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d1372-ae9e-40ed-baf1-078b2fb390b0 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:32.606867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e08d1372-ae9e-40ed-baf1-078b2fb390b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:32.812323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d1372-ae9e-40ed-baf1-078b2fb390b0 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:32.819014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:32.819711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:32.819981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:32.820268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] PATH_INFO: `/v3/projects/18a0836c389640a4bfeefaad3d93bc8a` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:32.821629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a0836c389640a4bfeefaad3d93bc8a)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:32.823575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:32.889606 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a0836c389640a4bfeefaad3d93bc8a', 'request_id': 'req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:32.896945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a0836c389640a4bfeefaad3d93bc8a', 'request_id': 'req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:32.907344 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a0836c389640a4bfeefaad3d93bc8a', 'request_id': 'req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:32.939705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] The token cache is being invalidate because project 18a0836c389640a4bfeefaad3d93bc8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:32.940062 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:32.940841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8e6e0f63-87e4-4f1a-b070-bdb7f5d08ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:32.952444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 941/1864] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:32 2022] DELETE /identity/v3/projects/18a0836c389640a4bfeefaad3d93bc8a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:42.545665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:DELETE=8,keystone:SELECT=76,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:42.936428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:51.451270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:51.452751 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:51.467406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b781f-20a2-4b65-8bcb-57e5215afa52 service nova] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:51.536572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:51.537281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:51.537629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:51.537912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] PATH_INFO: `/v3/users/900f27b4d0374ad79a2e3250a0c0a012` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:51.539202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=900f27b4d0374ad79a2e3250a0c0a012)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:51.541228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:51.590291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '900f27b4d0374ad79a2e3250a0c0a012'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:51.597064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '900f27b4d0374ad79a2e3250a0c0a012'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:51.598021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '900f27b4d0374ad79a2e3250a0c0a012'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:51.598689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '900f27b4d0374ad79a2e3250a0c0a012'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:51.606681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '900f27b4d0374ad79a2e3250a0c0a012'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:51.624748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 924/1865] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:42:51 2022] DELETE /identity/v3/users/900f27b4d0374ad79a2e3250a0c0a012 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:51.629379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6f15b-6307-45f6-8997-1ec4d0695e10 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:51.629856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6f15b-6307-45f6-8997-1ec4d0695e10 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:51.629950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6f15b-6307-45f6-8997-1ec4d0695e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:51.648768 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-79e6f15b-6307-45f6-8997-1ec4d0695e10 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:51.649966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-79e6f15b-6307-45f6-8997-1ec4d0695e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:51.712995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 942/1866] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:51.723123 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:51.724765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:51.784195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98341b-b228-4bbc-8d85-0acee90577c3 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:51.787766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b9537-6ec9-47ab-8b94-ffd90ab0ef2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:51.788409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b9537-6ec9-47ab-8b94-ffd90ab0ef2f service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:51.788693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b9537-6ec9-47ab-8b94-ffd90ab0ef2f service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:51.788983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b9537-6ec9-47ab-8b94-ffd90ab0ef2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:51.798773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b9537-6ec9-47ab-8b94-ffd90ab0ef2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:51.800229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b9537-6ec9-47ab-8b94-ffd90ab0ef2f service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:51.809996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 925/1867] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:42:52.081699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6f15b-6307-45f6-8997-1ec4d0695e10 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:52.083240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-79e6f15b-6307-45f6-8997-1ec4d0695e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:52.096026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6f15b-6307-45f6-8997-1ec4d0695e10 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:52.100629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:52.101264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.101543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.101820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] PATH_INFO: `/v3/projects/d23b6c045d6843beaaeee602786f2737` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.102985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23b6c045d6843beaaeee602786f2737)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:52.106037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:52.154033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23b6c045d6843beaaeee602786f2737', 'request_id': 'req-2a21c557-40a9-4263-b247-4280d838be7f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:52.159459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23b6c045d6843beaaeee602786f2737', 'request_id': 'req-2a21c557-40a9-4263-b247-4280d838be7f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:52.167589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23b6c045d6843beaaeee602786f2737', 'request_id': 'req-2a21c557-40a9-4263-b247-4280d838be7f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:52.178386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] The token cache is being invalidate because project d23b6c045d6843beaaeee602786f2737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:42:52.178662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:52.179295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-2a21c557-40a9-4263-b247-4280d838be7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:42:52.180802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 943/1868] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:42:52 2022] DELETE /identity/v3/projects/d23b6c045d6843beaaeee602786f2737 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:52.186487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc911-69ea-45b2-a06b-a4974cd8e1fd None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.186776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc911-69ea-45b2-a06b-a4974cd8e1fd None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.187056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc911-69ea-45b2-a06b-a4974cd8e1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.206305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-e8cdc911-69ea-45b2-a06b-a4974cd8e1fd None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:52.207500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e8cdc911-69ea-45b2-a06b-a4974cd8e1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:52.224583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:52.271717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 926/1869] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:52.280287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf483c5-729e-4d5e-89c3-6f1098366191 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.280614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf483c5-729e-4d5e-89c3-6f1098366191 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.280896 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf483c5-729e-4d5e-89c3-6f1098366191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.302540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-adf483c5-729e-4d5e-89c3-6f1098366191 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:52.303770 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-adf483c5-729e-4d5e-89c3-6f1098366191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:52.313957 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 944/1870] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:52.319620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc911-69ea-45b2-a06b-a4974cd8e1fd None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:52.330391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc911-69ea-45b2-a06b-a4974cd8e1fd None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:52.335288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5c90e-0676-475f-a963-98f50e7bd09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:52.336133 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5c90e-0676-475f-a963-98f50e7bd09c admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.336553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5c90e-0676-475f-a963-98f50e7bd09c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.340396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5c90e-0676-475f-a963-98f50e7bd09c admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.340396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5c90e-0676-475f-a963-98f50e7bd09c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:42:52.340396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5c90e-0676-475f-a963-98f50e7bd09c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:52.340396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5c90e-0676-475f-a963-98f50e7bd09c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:52.363422 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 927/1871] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:42:52.368951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf483c5-729e-4d5e-89c3-6f1098366191 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:52.386960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf483c5-729e-4d5e-89c3-6f1098366191 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:52.393085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f07f5-2554-46fc-a877-00f703409978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:52.393710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f07f5-2554-46fc-a877-00f703409978 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.393972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f07f5-2554-46fc-a877-00f703409978 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.394263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f07f5-2554-46fc-a877-00f703409978 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.394872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f07f5-2554-46fc-a877-00f703409978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:52.396809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f07f5-2554-46fc-a877-00f703409978 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:52.428372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 945/1872] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:42:52 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:52.429679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5c90e-0676-475f-a963-98f50e7bd09c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:52.440068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5c90e-0676-475f-a963-98f50e7bd09c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:52.497224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e330c-0802-4a43-a904-625a5bc68429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:52.497870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e330c-0802-4a43-a904-625a5bc68429 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.498104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e330c-0802-4a43-a904-625a5bc68429 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.498379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e330c-0802-4a43-a904-625a5bc68429 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.499025 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e330c-0802-4a43-a904-625a5bc68429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:52.501000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e330c-0802-4a43-a904-625a5bc68429 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:52.521735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 928/1873] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:42:52 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:42:52.526400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f07f5-2554-46fc-a877-00f703409978 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:52.536542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f07f5-2554-46fc-a877-00f703409978 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:52.541172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa33332-e532-4a17-a3a7-b79f3a93ce3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:52.541902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa33332-e532-4a17-a3a7-b79f3a93ce3b admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.542160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa33332-e532-4a17-a3a7-b79f3a93ce3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.542435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa33332-e532-4a17-a3a7-b79f3a93ce3b admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.543111 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa33332-e532-4a17-a3a7-b79f3a93ce3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:52.544748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa33332-e532-4a17-a3a7-b79f3a93ce3b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:52.554491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 946/1874] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:52.558905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e330c-0802-4a43-a904-625a5bc68429 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:52.570318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e330c-0802-4a43-a904-625a5bc68429 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:52.575401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945774c-1abc-4c6b-8de4-6e7e48ca40cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:52.576013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945774c-1abc-4c6b-8de4-6e7e48ca40cf admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.576298 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945774c-1abc-4c6b-8de4-6e7e48ca40cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.577931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945774c-1abc-4c6b-8de4-6e7e48ca40cf admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.577931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945774c-1abc-4c6b-8de4-6e7e48ca40cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:52.578984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945774c-1abc-4c6b-8de4-6e7e48ca40cf admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:52.593573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 929/1875] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:42:52.594733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa33332-e532-4a17-a3a7-b79f3a93ce3b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:52.605643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa33332-e532-4a17-a3a7-b79f3a93ce3b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:52.610879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dda5a-51e9-449d-ba14-e41126638700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:52.611612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dda5a-51e9-449d-ba14-e41126638700 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.611960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dda5a-51e9-449d-ba14-e41126638700 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.612316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dda5a-51e9-449d-ba14-e41126638700 admin admin] PATH_INFO: `/v3/projects/b3081eded62a439bae96ef2eacff6e65/users/98fed70e587f46a4b6bd8972ca5d6cb9/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.628842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dda5a-51e9-449d-ba14-e41126638700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3081eded62a439bae96ef2eacff6e65, user_id=98fed70e587f46a4b6bd8972ca5d6cb9, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:52.631594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dda5a-51e9-449d-ba14-e41126638700 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:52.646910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 947/1876] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:42:52 2022] PUT /identity/v3/projects/b3081eded62a439bae96ef2eacff6e65/users/98fed70e587f46a4b6bd8972ca5d6cb9/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:42:52.660982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d77f6e-ebca-415a-8569-c4fc12bad8d2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.661290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d77f6e-ebca-415a-8569-c4fc12bad8d2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.661655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d77f6e-ebca-415a-8569-c4fc12bad8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.691274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-43d77f6e-ebca-415a-8569-c4fc12bad8d2 None None] MFA Rules not processed for user `98fed70e587f46a4b6bd8972ca5d6cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:52.692540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-43d77f6e-ebca-415a-8569-c4fc12bad8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:52.760809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 930/1877] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:42:52.773682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dda5a-51e9-449d-ba14-e41126638700 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:42:52.775448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f10dda5a-51e9-449d-ba14-e41126638700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:52.799216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:52.839224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dda5a-51e9-449d-ba14-e41126638700 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:42:52.843688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ed318-a190-4321-87b8-94db5dfb38c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:42:52.844424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ed318-a190-4321-87b8-94db5dfb38c3 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:52.844890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ed318-a190-4321-87b8-94db5dfb38c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:52.845229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ed318-a190-4321-87b8-94db5dfb38c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:52.855724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ed318-a190-4321-87b8-94db5dfb38c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:42:52.857174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ed318-a190-4321-87b8-94db5dfb38c3 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:42:52.866170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 948/1878] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:42:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:42:55.950264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579cd4c-edbb-43b0-80b2-b2b0e6eb03c5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:42:55.950834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579cd4c-edbb-43b0-80b2-b2b0e6eb03c5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:42:55.950921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579cd4c-edbb-43b0-80b2-b2b0e6eb03c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:42:55.970707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6579cd4c-edbb-43b0-80b2-b2b0e6eb03c5 None None] MFA Rules not processed for user `ac5921418d40446c80892b04a3eee12d`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:42:55.971915 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6579cd4c-edbb-43b0-80b2-b2b0e6eb03c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:42:55.984828 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:42:56.030278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 931/1879] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:01.473248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee3cdb-630f-45b7-8a74-e4c3c8719d6c None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:01.473772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee3cdb-630f-45b7-8a74-e4c3c8719d6c None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:01.473873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee3cdb-630f-45b7-8a74-e4c3c8719d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:01.493788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f4ee3cdb-630f-45b7-8a74-e4c3c8719d6c None None] MFA Rules not processed for user `e0d4be0e0f744242a323ac9694aef9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:01.495016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f4ee3cdb-630f-45b7-8a74-e4c3c8719d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:01.548838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 949/1880] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:06.022770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=17 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:11.542059 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=42 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:13.097604 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecccc92-4f3d-4afb-87eb-8fac8be9fe35 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:13.098131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecccc92-4f3d-4afb-87eb-8fac8be9fe35 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:13.098131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecccc92-4f3d-4afb-87eb-8fac8be9fe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:13.118032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-0ecccc92-4f3d-4afb-87eb-8fac8be9fe35 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:13.119998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0ecccc92-4f3d-4afb-87eb-8fac8be9fe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:13.177679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 932/1881] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:13.229933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee3cdb-630f-45b7-8a74-e4c3c8719d6c None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:13.231508 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f4ee3cdb-630f-45b7-8a74-e4c3c8719d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:13.305232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee3cdb-630f-45b7-8a74-e4c3c8719d6c None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:13.310672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90092fa7-927d-497e-9a18-00e1510aab58 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:13.311282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90092fa7-927d-497e-9a18-00e1510aab58 service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:13.311561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90092fa7-927d-497e-9a18-00e1510aab58 service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:13.311829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90092fa7-927d-497e-9a18-00e1510aab58 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:13.320120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90092fa7-927d-497e-9a18-00e1510aab58 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:13.322387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90092fa7-927d-497e-9a18-00e1510aab58 service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:13.331552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 950/1882] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:43:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:43:19.992142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecccc92-4f3d-4afb-87eb-8fac8be9fe35 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:19.996878 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0ecccc92-4f3d-4afb-87eb-8fac8be9fe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:20.076369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecccc92-4f3d-4afb-87eb-8fac8be9fe35 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:20.080058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b26fc-e2e2-456e-a4d3-b39985f7c2d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:20.080779 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b26fc-e2e2-456e-a4d3-b39985f7c2d9 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:20.081128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b26fc-e2e2-456e-a4d3-b39985f7c2d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:20.081474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b26fc-e2e2-456e-a4d3-b39985f7c2d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:20.083515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-978b26fc-e2e2-456e-a4d3-b39985f7c2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:20.163288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b26fc-e2e2-456e-a4d3-b39985f7c2d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:20.164787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b26fc-e2e2-456e-a4d3-b39985f7c2d9 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:20.175816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 933/1883] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:43:23.327593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=32 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:28.242779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a6286-a1f0-4265-8a4f-69bb38076d4f None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:28.243385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a6286-a1f0-4265-8a4f-69bb38076d4f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:28.243385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a6286-a1f0-4265-8a4f-69bb38076d4f None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:28.245100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 951/1884] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:43:28.251044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553803-6a43-4c33-8c90-c6e6cb7b5938 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:28.251539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553803-6a43-4c33-8c90-c6e6cb7b5938 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:28.251631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553803-6a43-4c33-8c90-c6e6cb7b5938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:28.276957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-a5553803-6a43-4c33-8c90-c6e6cb7b5938 None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:28.278156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a5553803-6a43-4c33-8c90-c6e6cb7b5938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:28.327807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:28.356501 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 934/1885] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:28.366067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a6286-a1f0-4265-8a4f-69bb38076d4f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:28.367787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-691a6286-a1f0-4265-8a4f-69bb38076d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:28.426237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a6286-a1f0-4265-8a4f-69bb38076d4f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:28.429736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcfa83-51e5-4420-8cae-72d47dc5593f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cacfce5c9d07471cbe1713a4d6aa29f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:28.430335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcfa83-51e5-4420-8cae-72d47dc5593f service glance] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:28.430614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcfa83-51e5-4420-8cae-72d47dc5593f service glance] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:28.430880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcfa83-51e5-4420-8cae-72d47dc5593f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:28.440900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcfa83-51e5-4420-8cae-72d47dc5593f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:28.442328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcfa83-51e5-4420-8cae-72d47dc5593f service glance] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:28.452869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 952/1886] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:43:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:43:28.525793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc920c23-cf7c-48c1-a93c-884ec1128df9 None None] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:28.526563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc920c23-cf7c-48c1-a93c-884ec1128df9 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:28.527063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc920c23-cf7c-48c1-a93c-884ec1128df9 None None] PATH_INFO: `/` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:28.528458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 935/1887] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:43:28.533040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8aa6c7-0e92-44f7-b124-908bc33fd8ad None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:28.533324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8aa6c7-0e92-44f7-b124-908bc33fd8ad None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:28.533596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8aa6c7-0e92-44f7-b124-908bc33fd8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:28.554323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-cc8aa6c7-0e92-44f7-b124-908bc33fd8ad None None] MFA Rules not processed for user `ae4b023dbf114bf98d0e38ea7fec94fa`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:28.555519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-cc8aa6c7-0e92-44f7-b124-908bc33fd8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:28.562152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 953/1888] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:28.574615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc920c23-cf7c-48c1-a93c-884ec1128df9 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:28.576289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cc920c23-cf7c-48c1-a93c-884ec1128df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:28.590339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc920c23-cf7c-48c1-a93c-884ec1128df9 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:28.593875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48106b40-b2be-49dc-b9eb-c12b81bf97f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:28.594470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106b40-b2be-49dc-b9eb-c12b81bf97f5 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:28.594754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106b40-b2be-49dc-b9eb-c12b81bf97f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:28.595027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106b40-b2be-49dc-b9eb-c12b81bf97f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:28.601844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48106b40-b2be-49dc-b9eb-c12b81bf97f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:28.603265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48106b40-b2be-49dc-b9eb-c12b81bf97f5 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:28.614221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 936/1889] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:43:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:43:28.697265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8dccd-7b0c-46d0-ba74-7fa1030b308d None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:28.697627 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8dccd-7b0c-46d0-ba74-7fa1030b308d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:28.698482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8dccd-7b0c-46d0-ba74-7fa1030b308d None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:28.700038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 954/1890] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:43:28.705843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff67d0-43a5-41f1-adcb-3426222f6e94 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:28.706797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff67d0-43a5-41f1-adcb-3426222f6e94 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:28.706797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff67d0-43a5-41f1-adcb-3426222f6e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:28.727082 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d1ff67d0-43a5-41f1-adcb-3426222f6e94 None None] MFA Rules not processed for user `9647f8caa6cd4208800e2371fdf28301`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:28.728282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d1ff67d0-43a5-41f1-adcb-3426222f6e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:28.738100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 937/1891] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:43:28.756542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8dccd-7b0c-46d0-ba74-7fa1030b308d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:28.761189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8dccd-7b0c-46d0-ba74-7fa1030b308d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:28.764870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe24316-dac9-4209-9c48-199b0465245e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:28.765484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe24316-dac9-4209-9c48-199b0465245e service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:28.765745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe24316-dac9-4209-9c48-199b0465245e service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:28.766019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe24316-dac9-4209-9c48-199b0465245e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:28.777939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe24316-dac9-4209-9c48-199b0465245e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:28.779337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe24316-dac9-4209-9c48-199b0465245e service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:28.790290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 955/1892] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:43:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:43:31.558690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f23b68-309b-4533-ad40-41d9497f6570 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:31.559154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f23b68-309b-4533-ad40-41d9497f6570 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:31.559243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f23b68-309b-4533-ad40-41d9497f6570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:31.578530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-68f23b68-309b-4533-ad40-41d9497f6570 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:31.579704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-68f23b68-309b-4533-ad40-41d9497f6570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:31.586672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 938/1893] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:31.598720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe24316-dac9-4209-9c48-199b0465245e service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:31.610818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe24316-dac9-4209-9c48-199b0465245e service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:31.616146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa13455-27f8-40cf-a31d-d7b970da6f36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:31.616734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa13455-27f8-40cf-a31d-d7b970da6f36 service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:31.617020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa13455-27f8-40cf-a31d-d7b970da6f36 service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:31.617295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa13455-27f8-40cf-a31d-d7b970da6f36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:31.628380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa13455-27f8-40cf-a31d-d7b970da6f36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:31.629605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa13455-27f8-40cf-a31d-d7b970da6f36 service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:31.638888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 956/1894] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:43:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:43:32.172782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f23b68-309b-4533-ad40-41d9497f6570 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:32.247819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f23b68-309b-4533-ad40-41d9497f6570 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:32.252519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:32.253172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:32.253454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:32.253732 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] PATH_INFO: `/v3/users/98fed70e587f46a4b6bd8972ca5d6cb9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:32.254865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98fed70e587f46a4b6bd8972ca5d6cb9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:32.256854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:32.304161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98fed70e587f46a4b6bd8972ca5d6cb9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:32.312763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98fed70e587f46a4b6bd8972ca5d6cb9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:32.313472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98fed70e587f46a4b6bd8972ca5d6cb9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:32.314091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98fed70e587f46a4b6bd8972ca5d6cb9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:32.324318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98fed70e587f46a4b6bd8972ca5d6cb9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:32.342547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 939/1895] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:43:32 2022] DELETE /identity/v3/users/98fed70e587f46a4b6bd8972ca5d6cb9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:43:32.347119 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3dc4c-f53a-4fae-9fa4-f7047c3402d4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:32.347427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3dc4c-f53a-4fae-9fa4-f7047c3402d4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:32.347836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3dc4c-f53a-4fae-9fa4-f7047c3402d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:32.368639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-3db3dc4c-f53a-4fae-9fa4-f7047c3402d4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:32.370566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3db3dc4c-f53a-4fae-9fa4-f7047c3402d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:32.431853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 957/1896] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:43:32.442044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:32.443417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:32.489244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:32.505917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835eae1-cbf2-4904-a67e-759b6393bcd8 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:32.509508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149cfa-31d3-4ded-909e-303ad250bf98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:32.510123 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149cfa-31d3-4ded-909e-303ad250bf98 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:32.510401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149cfa-31d3-4ded-909e-303ad250bf98 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:32.510755 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149cfa-31d3-4ded-909e-303ad250bf98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:32.520159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149cfa-31d3-4ded-909e-303ad250bf98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:32.521630 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149cfa-31d3-4ded-909e-303ad250bf98 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:32.531917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 940/1897] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:43:32.822628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3dc4c-f53a-4fae-9fa4-f7047c3402d4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:32.824671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3db3dc4c-f53a-4fae-9fa4-f7047c3402d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:32.860775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3dc4c-f53a-4fae-9fa4-f7047c3402d4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:32.872014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:32.872738 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:32.873425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:32.873558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] PATH_INFO: `/v3/projects/b3081eded62a439bae96ef2eacff6e65` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:32.876018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3081eded62a439bae96ef2eacff6e65)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:32.878051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:32.940503 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3081eded62a439bae96ef2eacff6e65', 'request_id': 'req-cb93b377-0575-407d-a0dd-fd1d961008dd'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:32.951490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3081eded62a439bae96ef2eacff6e65', 'request_id': 'req-cb93b377-0575-407d-a0dd-fd1d961008dd'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:32.970009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3081eded62a439bae96ef2eacff6e65', 'request_id': 'req-cb93b377-0575-407d-a0dd-fd1d961008dd'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:32.971926 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:32.998428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] The token cache is being invalidate because project b3081eded62a439bae96ef2eacff6e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:43:32.998728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:32.999409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-cb93b377-0575-407d-a0dd-fd1d961008dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:33.001140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 958/1898] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:43:32 2022] DELETE /identity/v3/projects/b3081eded62a439bae96ef2eacff6e65 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:43:33.005898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3e427-9095-46e6-a3b6-5bcebc0cf40c None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.006398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3e427-9095-46e6-a3b6-5bcebc0cf40c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.006503 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3e427-9095-46e6-a3b6-5bcebc0cf40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.043562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-e0d3e427-9095-46e6-a3b6-5bcebc0cf40c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:33.044951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e0d3e427-9095-46e6-a3b6-5bcebc0cf40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:33.126640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 941/1899] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:33.136439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac365678-8d22-4660-91e5-a9c6a4c40f73 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.136825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac365678-8d22-4660-91e5-a9c6a4c40f73 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.137141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac365678-8d22-4660-91e5-a9c6a4c40f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.165927 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ac365678-8d22-4660-91e5-a9c6a4c40f73 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:33.167860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ac365678-8d22-4660-91e5-a9c6a4c40f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:33.186010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 959/1900] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:33.190243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3e427-9095-46e6-a3b6-5bcebc0cf40c None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:33.200870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3e427-9095-46e6-a3b6-5bcebc0cf40c None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:33.205756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf68b5-ae84-43ed-83ab-975691603684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:33.206355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf68b5-ae84-43ed-83ab-975691603684 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.206627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf68b5-ae84-43ed-83ab-975691603684 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.206922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf68b5-ae84-43ed-83ab-975691603684 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.207562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf68b5-ae84-43ed-83ab-975691603684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:43:33.208120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf68b5-ae84-43ed-83ab-975691603684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:33.209809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf68b5-ae84-43ed-83ab-975691603684 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:33.223435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 942/1901] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:43:33.227825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac365678-8d22-4660-91e5-a9c6a4c40f73 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:33.238066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac365678-8d22-4660-91e5-a9c6a4c40f73 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:33.242813 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83ae66-6379-4df6-97a9-42b36f3dfc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:33.243417 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83ae66-6379-4df6-97a9-42b36f3dfc4d admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.243699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83ae66-6379-4df6-97a9-42b36f3dfc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.243974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83ae66-6379-4df6-97a9-42b36f3dfc4d admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.245293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83ae66-6379-4df6-97a9-42b36f3dfc4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:33.247206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83ae66-6379-4df6-97a9-42b36f3dfc4d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:33.274571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 960/1902] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:43:33 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:43:33.278726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf68b5-ae84-43ed-83ab-975691603684 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:33.289652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf68b5-ae84-43ed-83ab-975691603684 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:33.357009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049eb1f-af4d-460b-8909-530852140b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:33.357666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049eb1f-af4d-460b-8909-530852140b3a admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.357998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049eb1f-af4d-460b-8909-530852140b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.358228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049eb1f-af4d-460b-8909-530852140b3a admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.359302 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049eb1f-af4d-460b-8909-530852140b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:33.361339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049eb1f-af4d-460b-8909-530852140b3a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:33.383478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 943/1903] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:43:33 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:43:33.388019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83ae66-6379-4df6-97a9-42b36f3dfc4d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:33.398589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83ae66-6379-4df6-97a9-42b36f3dfc4d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:33.403697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e187a-7a78-47cf-83ee-921df19e5f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:33.404333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e187a-7a78-47cf-83ee-921df19e5f96 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.404642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e187a-7a78-47cf-83ee-921df19e5f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.404944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e187a-7a78-47cf-83ee-921df19e5f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.405566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e187a-7a78-47cf-83ee-921df19e5f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:33.407191 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e187a-7a78-47cf-83ee-921df19e5f96 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:33.418582 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 961/1904] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:43:33.422692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049eb1f-af4d-460b-8909-530852140b3a admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:33.433687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049eb1f-af4d-460b-8909-530852140b3a admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:33.438375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd44470-a668-42da-b287-8c7a8b9c1ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:33.438927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd44470-a668-42da-b287-8c7a8b9c1ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.439202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd44470-a668-42da-b287-8c7a8b9c1ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.439488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd44470-a668-42da-b287-8c7a8b9c1ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.440115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd44470-a668-42da-b287-8c7a8b9c1ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:33.441733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd44470-a668-42da-b287-8c7a8b9c1ac1 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:33.453762 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 944/1905] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:43:33.458205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e187a-7a78-47cf-83ee-921df19e5f96 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:33.469522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e187a-7a78-47cf-83ee-921df19e5f96 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:33.474200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a4e58-2904-42a8-9c17-33a626dbc882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:33.474844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a4e58-2904-42a8-9c17-33a626dbc882 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.475120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a4e58-2904-42a8-9c17-33a626dbc882 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.475390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a4e58-2904-42a8-9c17-33a626dbc882 admin admin] PATH_INFO: `/v3/projects/84426482696940f5aa3ffb747382e178/users/6e61e196753147fd83ce6709b248085b/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.493273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a4e58-2904-42a8-9c17-33a626dbc882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84426482696940f5aa3ffb747382e178, user_id=6e61e196753147fd83ce6709b248085b, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:33.496141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a4e58-2904-42a8-9c17-33a626dbc882 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:33.510204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 962/1906] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:43:33 2022] PUT /identity/v3/projects/84426482696940f5aa3ffb747382e178/users/6e61e196753147fd83ce6709b248085b/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:43:33.521868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43dcc6-fa0c-43d1-9e11-d464d2164a76 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.522183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43dcc6-fa0c-43d1-9e11-d464d2164a76 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.522463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43dcc6-fa0c-43d1-9e11-d464d2164a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.552016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-dd43dcc6-fa0c-43d1-9e11-d464d2164a76 None None] MFA Rules not processed for user `6e61e196753147fd83ce6709b248085b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:33.553284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-dd43dcc6-fa0c-43d1-9e11-d464d2164a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:33.592489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:33.621379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 945/1907] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:33.632296 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a4e58-2904-42a8-9c17-33a626dbc882 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:33.634432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9e1a4e58-2904-42a8-9c17-33a626dbc882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:33.693833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a4e58-2904-42a8-9c17-33a626dbc882 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:33.697502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbf3f1-6cba-47e0-b75f-3715df0b3fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:33.698725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbf3f1-6cba-47e0-b75f-3715df0b3fbd service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:33.699008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbf3f1-6cba-47e0-b75f-3715df0b3fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:33.699289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbf3f1-6cba-47e0-b75f-3715df0b3fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:33.708037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbf3f1-6cba-47e0-b75f-3715df0b3fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:33.709500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbf3f1-6cba-47e0-b75f-3715df0b3fbd service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:33.719297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 963/1908] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:43:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:43:43.617594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=10 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:43.715060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:53.215323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43dcc6-fa0c-43d1-9e11-d464d2164a76 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:53.278466 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43dcc6-fa0c-43d1-9e11-d464d2164a76 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:53.283041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:53.283673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:53.283949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:53.284225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] PATH_INFO: `/v3/users/6e61e196753147fd83ce6709b248085b` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:53.285372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e61e196753147fd83ce6709b248085b)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:53.287308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:53.336058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e61e196753147fd83ce6709b248085b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:53.344711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e61e196753147fd83ce6709b248085b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:53.345493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e61e196753147fd83ce6709b248085b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:53.346152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e61e196753147fd83ce6709b248085b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:53.356969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e61e196753147fd83ce6709b248085b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:53.377076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 946/1909] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:43:53 2022] DELETE /identity/v3/users/6e61e196753147fd83ce6709b248085b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:43:53.381696 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe7e50-8805-4861-8ac1-2c24b7de5ce9 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:53.382211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe7e50-8805-4861-8ac1-2c24b7de5ce9 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:53.382364 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe7e50-8805-4861-8ac1-2c24b7de5ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:53.403276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-94fe7e50-8805-4861-8ac1-2c24b7de5ce9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:53.404543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-94fe7e50-8805-4861-8ac1-2c24b7de5ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:53.468311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 964/1910] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:53.478465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:53.480158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:53.543738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd90f-71a6-4fb5-9223-73e04bb0aa41 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:53.547557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43301a82-4496-4bcd-9542-4b4b953fba08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:53.548179 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43301a82-4496-4bcd-9542-4b4b953fba08 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:53.548491 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43301a82-4496-4bcd-9542-4b4b953fba08 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:53.548769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43301a82-4496-4bcd-9542-4b4b953fba08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:53.558533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43301a82-4496-4bcd-9542-4b4b953fba08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:53.560003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43301a82-4496-4bcd-9542-4b4b953fba08 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:53.570146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 947/1911] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:43:53.927014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe7e50-8805-4861-8ac1-2c24b7de5ce9 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:53.928835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-94fe7e50-8805-4861-8ac1-2c24b7de5ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:53.942484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe7e50-8805-4861-8ac1-2c24b7de5ce9 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:53.947173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:53.947809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:53.948275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:53.949031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] PATH_INFO: `/v3/projects/84426482696940f5aa3ffb747382e178` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:53.950176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84426482696940f5aa3ffb747382e178)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:53.952891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:54.009319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84426482696940f5aa3ffb747382e178', 'request_id': 'req-8d611b2d-c092-4b8e-a880-cdf239997405'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:54.015042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84426482696940f5aa3ffb747382e178', 'request_id': 'req-8d611b2d-c092-4b8e-a880-cdf239997405'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:54.022529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84426482696940f5aa3ffb747382e178', 'request_id': 'req-8d611b2d-c092-4b8e-a880-cdf239997405'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:54.034206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] The token cache is being invalidate because project 84426482696940f5aa3ffb747382e178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:43:54.034493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:54.035137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8d611b2d-c092-4b8e-a880-cdf239997405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:43:54.036673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 965/1912] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:43:53 2022] DELETE /identity/v3/projects/84426482696940f5aa3ffb747382e178 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:43:54.043349 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209558d8-0d95-4a78-8adb-3ac907fc9466 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.043646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209558d8-0d95-4a78-8adb-3ac907fc9466 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.043946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209558d8-0d95-4a78-8adb-3ac907fc9466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.063921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-209558d8-0d95-4a78-8adb-3ac907fc9466 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:54.065140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-209558d8-0d95-4a78-8adb-3ac907fc9466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:54.082233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:54.126574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 948/1913] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:54.134207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588de4d-7a29-4b11-86f3-bb55647c58c6 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.134490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588de4d-7a29-4b11-86f3-bb55647c58c6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.134786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588de4d-7a29-4b11-86f3-bb55647c58c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.154662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e588de4d-7a29-4b11-86f3-bb55647c58c6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:54.155892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e588de4d-7a29-4b11-86f3-bb55647c58c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:54.164128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 966/1914] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:54.169202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209558d8-0d95-4a78-8adb-3ac907fc9466 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:54.178860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209558d8-0d95-4a78-8adb-3ac907fc9466 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:54.184898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a369da-df48-4d0c-aaa9-c2e1f68f847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:54.185510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a369da-df48-4d0c-aaa9-c2e1f68f847c admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.185798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a369da-df48-4d0c-aaa9-c2e1f68f847c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.186076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a369da-df48-4d0c-aaa9-c2e1f68f847c admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.186734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a369da-df48-4d0c-aaa9-c2e1f68f847c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:43:54.187044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a369da-df48-4d0c-aaa9-c2e1f68f847c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:54.188719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a369da-df48-4d0c-aaa9-c2e1f68f847c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:54.200672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 949/1915] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:43:54.204911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588de4d-7a29-4b11-86f3-bb55647c58c6 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:54.214084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588de4d-7a29-4b11-86f3-bb55647c58c6 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:54.218882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ded93-bb00-4579-b486-3b5befe1e702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:54.219466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ded93-bb00-4579-b486-3b5befe1e702 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.219741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ded93-bb00-4579-b486-3b5befe1e702 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.220012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ded93-bb00-4579-b486-3b5befe1e702 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.220688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ded93-bb00-4579-b486-3b5befe1e702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:54.222565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ded93-bb00-4579-b486-3b5befe1e702 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:54.253399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 967/1916] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:43:54 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:43:54.254497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a369da-df48-4d0c-aaa9-c2e1f68f847c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:54.265395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a369da-df48-4d0c-aaa9-c2e1f68f847c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:54.320175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b95ee-8d7b-432f-9971-bac6338d68d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:54.320802 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b95ee-8d7b-432f-9971-bac6338d68d2 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.321077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b95ee-8d7b-432f-9971-bac6338d68d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.321348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b95ee-8d7b-432f-9971-bac6338d68d2 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.321971 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b95ee-8d7b-432f-9971-bac6338d68d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:54.324506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b95ee-8d7b-432f-9971-bac6338d68d2 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:54.349747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 950/1917] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:43:54 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:43:54.353553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ded93-bb00-4579-b486-3b5befe1e702 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:54.364667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ded93-bb00-4579-b486-3b5befe1e702 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:54.369436 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471dfc0-9137-4f29-a6d2-c80f75701de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:54.370485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471dfc0-9137-4f29-a6d2-c80f75701de9 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.370904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471dfc0-9137-4f29-a6d2-c80f75701de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.371214 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471dfc0-9137-4f29-a6d2-c80f75701de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.372078 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471dfc0-9137-4f29-a6d2-c80f75701de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:54.374063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471dfc0-9137-4f29-a6d2-c80f75701de9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:54.390532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 968/1918] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:43:54.392435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b95ee-8d7b-432f-9971-bac6338d68d2 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:54.404242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b95ee-8d7b-432f-9971-bac6338d68d2 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:54.410161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74e226-0b16-4444-9af9-26692568bcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:54.410773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74e226-0b16-4444-9af9-26692568bcbd admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.411055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74e226-0b16-4444-9af9-26692568bcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.411334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74e226-0b16-4444-9af9-26692568bcbd admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.411978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74e226-0b16-4444-9af9-26692568bcbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:54.413628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74e226-0b16-4444-9af9-26692568bcbd admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:54.424398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 951/1919] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:43:54.428559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471dfc0-9137-4f29-a6d2-c80f75701de9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:54.440398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471dfc0-9137-4f29-a6d2-c80f75701de9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:54.445199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3af2c-9cce-45f7-a463-30134bcf578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:54.445856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3af2c-9cce-45f7-a463-30134bcf578d admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.446137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3af2c-9cce-45f7-a463-30134bcf578d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.446411 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3af2c-9cce-45f7-a463-30134bcf578d admin admin] PATH_INFO: `/v3/projects/adc778cfb32049d5844da08c78ba014a/users/95515d28733f4c7e837eee5673346901/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.462056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3af2c-9cce-45f7-a463-30134bcf578d admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc778cfb32049d5844da08c78ba014a, user_id=95515d28733f4c7e837eee5673346901, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:54.464963 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3af2c-9cce-45f7-a463-30134bcf578d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:54.478869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 969/1920] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:43:54 2022] PUT /identity/v3/projects/adc778cfb32049d5844da08c78ba014a/users/95515d28733f4c7e837eee5673346901/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:43:54.484173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786480-8ea0-4d1a-9f61-0b84e2f484ba None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.484774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786480-8ea0-4d1a-9f61-0b84e2f484ba None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.485053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786480-8ea0-4d1a-9f61-0b84e2f484ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.506747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-27786480-8ea0-4d1a-9f61-0b84e2f484ba None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:54.507871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-27786480-8ea0-4d1a-9f61-0b84e2f484ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:54.567624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 952/1921] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:54.577814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3af2c-9cce-45f7-a463-30134bcf578d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:54.579687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3de3af2c-9cce-45f7-a463-30134bcf578d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:54.601844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:54.639438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3af2c-9cce-45f7-a463-30134bcf578d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:54.643156 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938b35a-45e5-40e5-b1a4-74fe7861fe2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:54.643762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938b35a-45e5-40e5-b1a4-74fe7861fe2a service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:54.644041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938b35a-45e5-40e5-b1a4-74fe7861fe2a service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:54.644314 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938b35a-45e5-40e5-b1a4-74fe7861fe2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:54.653115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938b35a-45e5-40e5-b1a4-74fe7861fe2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:54.654599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938b35a-45e5-40e5-b1a4-74fe7861fe2a service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:54.664130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 970/1922] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:43:56.771099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a485ed-443a-4f9c-8e67-f07f4792664d None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:56.771630 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a485ed-443a-4f9c-8e67-f07f4792664d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:56.772137 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a485ed-443a-4f9c-8e67-f07f4792664d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:56.796402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-57a485ed-443a-4f9c-8e67-f07f4792664d None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:56.797926 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-57a485ed-443a-4f9c-8e67-f07f4792664d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:56.825219 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:43:56.862352 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 953/1923] 10.222.0.13 () {58 vars in 967 bytes} [Thu Oct 20 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:56.877396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938b35a-45e5-40e5-b1a4-74fe7861fe2a service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:56.879393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5938b35a-45e5-40e5-b1a4-74fe7861fe2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:56.980864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938b35a-45e5-40e5-b1a4-74fe7861fe2a service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:56.989472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905124f3-26da-40df-8da9-ada343f95dbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:56.990771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905124f3-26da-40df-8da9-ada343f95dbe service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:56.991059 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905124f3-26da-40df-8da9-ada343f95dbe service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:56.991332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905124f3-26da-40df-8da9-ada343f95dbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:57.004589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905124f3-26da-40df-8da9-ada343f95dbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:57.006662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905124f3-26da-40df-8da9-ada343f95dbe service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:57.017294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 971/1924] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:43:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:43:58.783384 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765f5a7-4580-412f-8ef1-5dec437e179d None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:58.783823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765f5a7-4580-412f-8ef1-5dec437e179d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:58.784196 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765f5a7-4580-412f-8ef1-5dec437e179d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:58.816850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6765f5a7-4580-412f-8ef1-5dec437e179d None None] MFA Rules not processed for user `95515d28733f4c7e837eee5673346901`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:43:58.818099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6765f5a7-4580-412f-8ef1-5dec437e179d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:43:58.887856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 954/1925] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:43:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:43:58.892417 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905124f3-26da-40df-8da9-ada343f95dbe service swift] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:58.904529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905124f3-26da-40df-8da9-ada343f95dbe service swift] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:58.912678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee5ad-70e3-421f-8857-3240751d2f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:58.913283 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee5ad-70e3-421f-8857-3240751d2f11 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:58.913560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee5ad-70e3-421f-8857-3240751d2f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:58.913829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee5ad-70e3-421f-8857-3240751d2f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:58.914456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ee5ad-70e3-421f-8857-3240751d2f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:58.917083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ee5ad-70e3-421f-8857-3240751d2f11 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:58.950184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765f5a7-4580-412f-8ef1-5dec437e179d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:58.950373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 972/1926] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:43:58 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:43:58.969850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765f5a7-4580-412f-8ef1-5dec437e179d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:59.048331 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a81486-1c75-47a3-ad10-0c63de7a6711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:59.051793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a81486-1c75-47a3-ad10-0c63de7a6711 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:59.052081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a81486-1c75-47a3-ad10-0c63de7a6711 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:59.052386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a81486-1c75-47a3-ad10-0c63de7a6711 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:59.053183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a81486-1c75-47a3-ad10-0c63de7a6711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:59.055182 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a81486-1c75-47a3-ad10-0c63de7a6711 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:59.077763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 955/1927] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:43:58 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:43:59.081799 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee5ad-70e3-421f-8857-3240751d2f11 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:59.093159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee5ad-70e3-421f-8857-3240751d2f11 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:59.097800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd114751-24c1-4ce8-b955-7e5c9e14fa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:59.098413 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd114751-24c1-4ce8-b955-7e5c9e14fa70 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:59.098688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd114751-24c1-4ce8-b955-7e5c9e14fa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:59.098962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd114751-24c1-4ce8-b955-7e5c9e14fa70 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:59.099604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd114751-24c1-4ce8-b955-7e5c9e14fa70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:59.101269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd114751-24c1-4ce8-b955-7e5c9e14fa70 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:59.115534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 973/1928] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:43:59.119762 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a81486-1c75-47a3-ad10-0c63de7a6711 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:59.130512 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a81486-1c75-47a3-ad10-0c63de7a6711 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:59.135521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e4bbb-a96e-4b8a-bebf-3aa09c248c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:59.136889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e4bbb-a96e-4b8a-bebf-3aa09c248c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:59.137195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e4bbb-a96e-4b8a-bebf-3aa09c248c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:59.137472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e4bbb-a96e-4b8a-bebf-3aa09c248c4c admin admin] PATH_INFO: `/v3/projects/360cea05ebfc4d95b3b37a53f9673b3c/users/65bcd2a87f774ec9bd3d1f66c58eda08/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:59.154750 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e4bbb-a96e-4b8a-bebf-3aa09c248c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=360cea05ebfc4d95b3b37a53f9673b3c, user_id=65bcd2a87f774ec9bd3d1f66c58eda08, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:59.157558 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e4bbb-a96e-4b8a-bebf-3aa09c248c4c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:59.172164 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 956/1929] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:43:59 2022] PUT /identity/v3/projects/360cea05ebfc4d95b3b37a53f9673b3c/users/65bcd2a87f774ec9bd3d1f66c58eda08/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:43:59.176147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd114751-24c1-4ce8-b955-7e5c9e14fa70 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:59.187646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd114751-24c1-4ce8-b955-7e5c9e14fa70 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:59.246458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa63d0-681a-4904-8a59-b474bd4c9415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:59.247076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa63d0-681a-4904-8a59-b474bd4c9415 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:59.247356 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa63d0-681a-4904-8a59-b474bd4c9415 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:59.247629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa63d0-681a-4904-8a59-b474bd4c9415 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:59.248276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa63d0-681a-4904-8a59-b474bd4c9415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:59.250020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa63d0-681a-4904-8a59-b474bd4c9415 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:59.264080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 974/1930] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:43:59.268383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e4bbb-a96e-4b8a-bebf-3aa09c248c4c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:43:59.279571 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e4bbb-a96e-4b8a-bebf-3aa09c248c4c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:43:59.284664 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bfa5e-9a16-43d6-a1bd-e292ae495103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:43:59.285334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bfa5e-9a16-43d6-a1bd-e292ae495103 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:43:59.285613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bfa5e-9a16-43d6-a1bd-e292ae495103 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:43:59.286077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bfa5e-9a16-43d6-a1bd-e292ae495103 admin admin] PATH_INFO: `/v3/projects/360cea05ebfc4d95b3b37a53f9673b3c/users/65bcd2a87f774ec9bd3d1f66c58eda08/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:43:59.288013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bfa5e-9a16-43d6-a1bd-e292ae495103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=360cea05ebfc4d95b3b37a53f9673b3c, user_id=65bcd2a87f774ec9bd3d1f66c58eda08, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:43:59.290770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bfa5e-9a16-43d6-a1bd-e292ae495103 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:43:59.302462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 957/1931] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:43:59 2022] PUT /identity/v3/projects/360cea05ebfc4d95b3b37a53f9673b3c/users/65bcd2a87f774ec9bd3d1f66c58eda08/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:44:00.924010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa63d0-681a-4904-8a59-b474bd4c9415 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:44:00.926076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e7aa63d0-681a-4904-8a59-b474bd4c9415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:00.959521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:44:01.047267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa63d0-681a-4904-8a59-b474bd4c9415 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:44:01.164104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc18a78229064e8d8ea4321c3c23c4ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:44:01.165165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:01.165560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:01.165962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:01.168129 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:01.183367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:44:01.185697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:44:01.208054 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 975/1932] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:44:03.787846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63658f4-78b2-4f32-95db-55cface04af8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:03.788379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63658f4-78b2-4f32-95db-55cface04af8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:03.788473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63658f4-78b2-4f32-95db-55cface04af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:03.820159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d63658f4-78b2-4f32-95db-55cface04af8 None None] MFA Rules not processed for user `65bcd2a87f774ec9bd3d1f66c58eda08`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:44:03.821320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d63658f4-78b2-4f32-95db-55cface04af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:03.826877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:44:03.893978 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 958/1933] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:44:03.906607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:44:03.908485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:03.976856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd90f5-8912-49e7-80cc-908c9f01d478 service placement] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:44:03.990000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56af501-fab8-4575-843f-1e63bd5c6098 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cacfce5c9d07471cbe1713a4d6aa29f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:44:03.990000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56af501-fab8-4575-843f-1e63bd5c6098 service glance] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:03.990400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56af501-fab8-4575-843f-1e63bd5c6098 service glance] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:03.990778 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56af501-fab8-4575-843f-1e63bd5c6098 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:04.062442 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56af501-fab8-4575-843f-1e63bd5c6098 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:44:04.064274 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56af501-fab8-4575-843f-1e63bd5c6098 service glance] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:44:04.074876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 976/1934] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:44:04.215837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eab814-2177-4e0f-81fd-e7aedc849f26 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:04.216221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eab814-2177-4e0f-81fd-e7aedc849f26 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:04.216610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eab814-2177-4e0f-81fd-e7aedc849f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:04.237989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-56eab814-2177-4e0f-81fd-e7aedc849f26 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:44:04.239383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-56eab814-2177-4e0f-81fd-e7aedc849f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:04.301280 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 959/1935] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:44:04.314816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56af501-fab8-4575-843f-1e63bd5c6098 service glance] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:44:04.319226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:44:04.391369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56af501-fab8-4575-843f-1e63bd5c6098 service glance] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:44:04.398710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335de114-2c55-4ca8-8e8c-a75db239ae8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:44:04.399405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335de114-2c55-4ca8-8e8c-a75db239ae8b service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:04.399757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335de114-2c55-4ca8-8e8c-a75db239ae8b service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:04.400104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335de114-2c55-4ca8-8e8c-a75db239ae8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:04.416373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335de114-2c55-4ca8-8e8c-a75db239ae8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:44:04.416373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335de114-2c55-4ca8-8e8c-a75db239ae8b service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:44:04.421199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 977/1936] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:44:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:44:13.641077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4427a75-fafa-40ed-b246-6e6dfc3fa72f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:13.641610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4427a75-fafa-40ed-b246-6e6dfc3fa72f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:13.641724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4427a75-fafa-40ed-b246-6e6dfc3fa72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:13.662066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d4427a75-fafa-40ed-b246-6e6dfc3fa72f None None] MFA Rules not processed for user `95515d28733f4c7e837eee5673346901`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:44:13.663255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d4427a75-fafa-40ed-b246-6e6dfc3fa72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:13.670233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 960/1937] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:44:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:44:14.422667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=31 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:44:20.179743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc7163-f561-4ca3-a1e0-4e78b59fb022 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:20.180332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc7163-f561-4ca3-a1e0-4e78b59fb022 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:20.180969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc7163-f561-4ca3-a1e0-4e78b59fb022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:20.202305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-5ebc7163-f561-4ca3-a1e0-4e78b59fb022 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:44:20.203492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5ebc7163-f561-4ca3-a1e0-4e78b59fb022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:20.210972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 978/1938] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:44:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:44:20.231453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4427a75-fafa-40ed-b246-6e6dfc3fa72f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:44:20.249540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4427a75-fafa-40ed-b246-6e6dfc3fa72f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:44:20.256033 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ef9f-cb9f-4feb-88cf-306cdac90e99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:44:20.256722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ef9f-cb9f-4feb-88cf-306cdac90e99 service swift] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:20.257001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ef9f-cb9f-4feb-88cf-306cdac90e99 service swift] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:20.257483 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ef9f-cb9f-4feb-88cf-306cdac90e99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:20.273833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ef9f-cb9f-4feb-88cf-306cdac90e99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:44:20.275548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ef9f-cb9f-4feb-88cf-306cdac90e99 service swift] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:44:20.290507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 961/1939] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:44:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:44:24.638021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bebc55-7966-44ef-b8fa-7cd09831748e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:24.638520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bebc55-7966-44ef-b8fa-7cd09831748e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:24.638631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bebc55-7966-44ef-b8fa-7cd09831748e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:24.659329 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-29bebc55-7966-44ef-b8fa-7cd09831748e None None] MFA Rules not processed for user `ae94a3ece5f644c6893589a1df84d1c6`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:44:24.660889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-29bebc55-7966-44ef-b8fa-7cd09831748e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:24.723149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 979/1940] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:44:30.281577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=61 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:44:34.715524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=28 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:44:41.509128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06ad75-06da-4c5f-a14c-3f5057f61394 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:41.509619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06ad75-06da-4c5f-a14c-3f5057f61394 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:41.509729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06ad75-06da-4c5f-a14c-3f5057f61394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:41.529391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-cf06ad75-06da-4c5f-a14c-3f5057f61394 None None] MFA Rules not processed for user `e0d4be0e0f744242a323ac9694aef9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:44:41.530596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cf06ad75-06da-4c5f-a14c-3f5057f61394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:41.585909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 962/1941] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:44:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:44:50.668843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bebc55-7966-44ef-b8fa-7cd09831748e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:44:50.670375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-29bebc55-7966-44ef-b8fa-7cd09831748e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:50.727360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bebc55-7966-44ef-b8fa-7cd09831748e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:44:50.730923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5a246-2b75-43f8-aebe-ef33e8e5fddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:44:50.731527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5a246-2b75-43f8-aebe-ef33e8e5fddc service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:50.731831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5a246-2b75-43f8-aebe-ef33e8e5fddc service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:50.732105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5a246-2b75-43f8-aebe-ef33e8e5fddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:50.742623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5a246-2b75-43f8-aebe-ef33e8e5fddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:44:50.744076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5a246-2b75-43f8-aebe-ef33e8e5fddc service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:44:50.755063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 980/1942] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:44:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:44:51.578984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=25 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:44:59.792687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad56f1-add2-4696-84a8-b4ab3cd65722 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:44:59.793325 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad56f1-add2-4696-84a8-b4ab3cd65722 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:44:59.793325 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad56f1-add2-4696-84a8-b4ab3cd65722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:44:59.812614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b1ad56f1-add2-4696-84a8-b4ab3cd65722 None None] MFA Rules not processed for user `9fc72f6965bb4814a5d85bd04769e4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:44:59.813797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b1ad56f1-add2-4696-84a8-b4ab3cd65722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:44:59.866806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 963/1943] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:45:00.750266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=26 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:45:09.860044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=25 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:45:46.756711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d0c9d-6221-4615-a37e-d558a6cb7481 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:45:46.757215 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d0c9d-6221-4615-a37e-d558a6cb7481 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:45:46.757312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d0c9d-6221-4615-a37e-d558a6cb7481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:45:46.777244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4f4d0c9d-6221-4615-a37e-d558a6cb7481 None None] MFA Rules not processed for user `ac5921418d40446c80892b04a3eee12d`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:45:46.778425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4f4d0c9d-6221-4615-a37e-d558a6cb7481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:45:46.832759 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 981/1944] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:45:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:45:48.987046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc05d-6b5e-40d6-8c99-2e952f4b78a9 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:45:48.987579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc05d-6b5e-40d6-8c99-2e952f4b78a9 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:45:48.987579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc05d-6b5e-40d6-8c99-2e952f4b78a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:45:49.014190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-02cfc05d-6b5e-40d6-8c99-2e952f4b78a9 None None] MFA Rules not processed for user `4d74d7fd62cd464183f9df4ca581833c`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:45:49.018373 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-02cfc05d-6b5e-40d6-8c99-2e952f4b78a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:45:49.082274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 964/1945] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:45:49.693984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d0c9d-6221-4615-a37e-d558a6cb7481 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:45:49.704265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d0c9d-6221-4615-a37e-d558a6cb7481 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:45:49.707856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca63c1-5c29-4c98-af2a-186e5dfe237d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:45:49.708493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca63c1-5c29-4c98-af2a-186e5dfe237d service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:45:49.708777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca63c1-5c29-4c98-af2a-186e5dfe237d service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:45:49.709056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca63c1-5c29-4c98-af2a-186e5dfe237d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:45:49.710754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-24ca63c1-5c29-4c98-af2a-186e5dfe237d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:45:49.784319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca63c1-5c29-4c98-af2a-186e5dfe237d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:45:49.785831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca63c1-5c29-4c98-af2a-186e5dfe237d service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:45:49.797688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 982/1946] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:45:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:45:51.173922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc05d-6b5e-40d6-8c99-2e952f4b78a9 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:45:51.183677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc05d-6b5e-40d6-8c99-2e952f4b78a9 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:45:51.188005 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102339f2-4420-49c9-9f35-d9c8828f3933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:45:51.188630 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102339f2-4420-49c9-9f35-d9c8828f3933 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:45:51.188912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102339f2-4420-49c9-9f35-d9c8828f3933 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:45:51.189185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102339f2-4420-49c9-9f35-d9c8828f3933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:45:51.190863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-102339f2-4420-49c9-9f35-d9c8828f3933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:45:51.204465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102339f2-4420-49c9-9f35-d9c8828f3933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:45:51.205891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102339f2-4420-49c9-9f35-d9c8828f3933 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:45:51.216696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 965/1947] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:45:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:45:52.544858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731ae54-6dc2-48ff-9d8b-72515b089cd2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:45:52.547995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731ae54-6dc2-48ff-9d8b-72515b089cd2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:45:52.551835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731ae54-6dc2-48ff-9d8b-72515b089cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:45:52.589594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-3731ae54-6dc2-48ff-9d8b-72515b089cd2 None None] MFA Rules not processed for user `ae94a3ece5f644c6893589a1df84d1c6`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:45:52.591438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3731ae54-6dc2-48ff-9d8b-72515b089cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:45:52.600440 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 983/1948] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:46:01.211760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=31 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:02.574228 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=62 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:04.795654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8454a-53c9-4312-9627-6414be82de4f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:04.796899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8454a-53c9-4312-9627-6414be82de4f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:04.797072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8454a-53c9-4312-9627-6414be82de4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:04.816520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-21a8454a-53c9-4312-9627-6414be82de4f None None] MFA Rules not processed for user `e0d4be0e0f744242a323ac9694aef9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:04.817710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-21a8454a-53c9-4312-9627-6414be82de4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:04.824523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 966/1949] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:46:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:46:06.007891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731ae54-6dc2-48ff-9d8b-72515b089cd2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:06.009339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3731ae54-6dc2-48ff-9d8b-72515b089cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:06.025511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731ae54-6dc2-48ff-9d8b-72515b089cd2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:06.032897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21869384-b49e-409d-96e6-05717a957b85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:06.033509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21869384-b49e-409d-96e6-05717a957b85 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:06.033787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21869384-b49e-409d-96e6-05717a957b85 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:06.034061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21869384-b49e-409d-96e6-05717a957b85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:06.035792 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-21869384-b49e-409d-96e6-05717a957b85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:06.046244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21869384-b49e-409d-96e6-05717a957b85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:06.047809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21869384-b49e-409d-96e6-05717a957b85 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:06.057643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 984/1950] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:06.648396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8454a-53c9-4312-9627-6414be82de4f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:06.649824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-21a8454a-53c9-4312-9627-6414be82de4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:06.665516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8454a-53c9-4312-9627-6414be82de4f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:06.671669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced0523-e42f-4714-ab60-5283e3a67dbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:06.672280 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced0523-e42f-4714-ab60-5283e3a67dbb service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:06.672598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced0523-e42f-4714-ab60-5283e3a67dbb service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:06.672877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced0523-e42f-4714-ab60-5283e3a67dbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:06.674588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9ced0523-e42f-4714-ab60-5283e3a67dbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:06.685323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced0523-e42f-4714-ab60-5283e3a67dbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:06.687699 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced0523-e42f-4714-ab60-5283e3a67dbb service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:06.699128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 967/1951] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:16.053528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=6 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:16.694327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=9 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:16.762252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21869384-b49e-409d-96e6-05717a957b85 service nova] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:16.773146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21869384-b49e-409d-96e6-05717a957b85 service nova] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:16.776812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0ddfd-89d7-4f75-b467-3270c9fe697c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:16.777419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0ddfd-89d7-4f75-b467-3270c9fe697c service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:16.777695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0ddfd-89d7-4f75-b467-3270c9fe697c service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:16.777994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0ddfd-89d7-4f75-b467-3270c9fe697c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:16.779746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a5d0ddfd-89d7-4f75-b467-3270c9fe697c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:16.789550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0ddfd-89d7-4f75-b467-3270c9fe697c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:16.790996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0ddfd-89d7-4f75-b467-3270c9fe697c service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:16.799365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 985/1952] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:20.189469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0ddfd-89d7-4f75-b467-3270c9fe697c service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:20.190108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5b6a5-3915-4d26-a6a7-79eddc2d4ad2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:20.190452 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5b6a5-3915-4d26-a6a7-79eddc2d4ad2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:20.190617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5b6a5-3915-4d26-a6a7-79eddc2d4ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:20.203437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0ddfd-89d7-4f75-b467-3270c9fe697c service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:20.209001 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7690a4-0082-4c70-8675-48f9e92520f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:20.209841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7690a4-0082-4c70-8675-48f9e92520f0 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:20.210125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7690a4-0082-4c70-8675-48f9e92520f0 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:20.210405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7690a4-0082-4c70-8675-48f9e92520f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:20.212043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-05e5b6a5-3915-4d26-a6a7-79eddc2d4ad2 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:20.212158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2d7690a4-0082-4c70-8675-48f9e92520f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:20.213357 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-05e5b6a5-3915-4d26-a6a7-79eddc2d4ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:20.221050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 968/1953] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:46:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:46:20.228330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7690a4-0082-4c70-8675-48f9e92520f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:20.230284 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7690a4-0082-4c70-8675-48f9e92520f0 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:20.242664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 986/1954] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:20.270368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5b6a5-3915-4d26-a6a7-79eddc2d4ad2 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:20.285168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5b6a5-3915-4d26-a6a7-79eddc2d4ad2 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:20.290368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d07ad1-6c7a-4a82-9a53-662048224ee2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:20.290996 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d07ad1-6c7a-4a82-9a53-662048224ee2 service swift] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:20.291275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d07ad1-6c7a-4a82-9a53-662048224ee2 service swift] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:20.291548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d07ad1-6c7a-4a82-9a53-662048224ee2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:20.300997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d07ad1-6c7a-4a82-9a53-662048224ee2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:20.302844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d07ad1-6c7a-4a82-9a53-662048224ee2 service swift] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:20.312079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 969/1955] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:46:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:46:30.238417 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=12 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:30.307900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=9 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:30.692008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14fae3-1938-4ed1-b9e9-f165aae0088e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:30.692586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14fae3-1938-4ed1-b9e9-f165aae0088e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:30.693338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14fae3-1938-4ed1-b9e9-f165aae0088e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:30.713435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-7f14fae3-1938-4ed1-b9e9-f165aae0088e None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:30.714722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7f14fae3-1938-4ed1-b9e9-f165aae0088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:30.723229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 987/1956] 10.222.0.13 () {58 vars in 967 bytes} [Thu Oct 20 14:46:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:46:30.737482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d07ad1-6c7a-4a82-9a53-662048224ee2 service swift] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:30.747891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d07ad1-6c7a-4a82-9a53-662048224ee2 service swift] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:30.753036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716716c-f21e-43a7-9775-028a7ddfb23b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:30.753638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716716c-f21e-43a7-9775-028a7ddfb23b service swift] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:30.753920 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716716c-f21e-43a7-9775-028a7ddfb23b service swift] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:30.754195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716716c-f21e-43a7-9775-028a7ddfb23b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:30.763668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716716c-f21e-43a7-9775-028a7ddfb23b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:30.765513 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716716c-f21e-43a7-9775-028a7ddfb23b service swift] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:30.773650 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 970/1957] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:46:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:46:36.682177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14fae3-1938-4ed1-b9e9-f165aae0088e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:36.699344 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14fae3-1938-4ed1-b9e9-f165aae0088e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:36.709804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c89df-a42f-4f4a-9f2e-1dc6558dda58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:36.710643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c89df-a42f-4f4a-9f2e-1dc6558dda58 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:36.711279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c89df-a42f-4f4a-9f2e-1dc6558dda58 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:36.711714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c89df-a42f-4f4a-9f2e-1dc6558dda58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:36.714005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8a7c89df-a42f-4f4a-9f2e-1dc6558dda58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:36.735291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c89df-a42f-4f4a-9f2e-1dc6558dda58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:36.739681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c89df-a42f-4f4a-9f2e-1dc6558dda58 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:36.759141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 988/1958] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:37.018492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716716c-f21e-43a7-9775-028a7ddfb23b service swift] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:37.030193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716716c-f21e-43a7-9775-028a7ddfb23b service swift] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:37.035451 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b97fb-a16e-4440-baf5-b42e25dd53d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc18a78229064e8d8ea4321c3c23c4ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:37.036064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b97fb-a16e-4440-baf5-b42e25dd53d7 service placement] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:37.036372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b97fb-a16e-4440-baf5-b42e25dd53d7 service placement] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:37.036656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b97fb-a16e-4440-baf5-b42e25dd53d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:37.038370 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a85b97fb-a16e-4440-baf5-b42e25dd53d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:37.050202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b97fb-a16e-4440-baf5-b42e25dd53d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:37.051809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b97fb-a16e-4440-baf5-b42e25dd53d7 service placement] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:37.062830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 971/1959] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:37.963836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c89df-a42f-4f4a-9f2e-1dc6558dda58 service nova] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:38.030899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c89df-a42f-4f4a-9f2e-1dc6558dda58 service nova] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:38.035326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:38.035952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:38.036233 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:38.036541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] PATH_INFO: `/v3/users/95515d28733f4c7e837eee5673346901` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:38.037655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95515d28733f4c7e837eee5673346901)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:38.039586 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:38.089493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95515d28733f4c7e837eee5673346901'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.097040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95515d28733f4c7e837eee5673346901'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.097726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95515d28733f4c7e837eee5673346901'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.098334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95515d28733f4c7e837eee5673346901'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.106875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95515d28733f4c7e837eee5673346901'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.124663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 989/1960] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:46:37 2022] DELETE /identity/v3/users/95515d28733f4c7e837eee5673346901 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:46:38.412911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b97fb-a16e-4440-baf5-b42e25dd53d7 service placement] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:38.423756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b97fb-a16e-4440-baf5-b42e25dd53d7 service placement] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:38.488621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:38.489596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:38.489975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:38.490496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] PATH_INFO: `/v3/projects/adc778cfb32049d5844da08c78ba014a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:38.491774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adc778cfb32049d5844da08c78ba014a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:38.493884 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:38.551168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adc778cfb32049d5844da08c78ba014a', 'request_id': 'req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.560990 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adc778cfb32049d5844da08c78ba014a', 'request_id': 'req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.569916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adc778cfb32049d5844da08c78ba014a', 'request_id': 'req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.583848 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] The token cache is being invalidate because project adc778cfb32049d5844da08c78ba014a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:46:38.584494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.585364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.587264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 972/1961] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:46:38 2022] DELETE /identity/v3/projects/adc778cfb32049d5844da08c78ba014a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:46:38.592764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:38.594422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:38.610112 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c70aae-e7d5-4e7e-bd94-bdaa10066d41 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:38.667569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:38.668124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:38.668430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:38.668705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] PATH_INFO: `/v3/users/65bcd2a87f774ec9bd3d1f66c58eda08` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:38.670118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65bcd2a87f774ec9bd3d1f66c58eda08)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:38.672062 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:38.673613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:38.723216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65bcd2a87f774ec9bd3d1f66c58eda08'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.730477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65bcd2a87f774ec9bd3d1f66c58eda08'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.731147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65bcd2a87f774ec9bd3d1f66c58eda08'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.731774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65bcd2a87f774ec9bd3d1f66c58eda08'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.740461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26982e05-47e8-4568-b6d4-f27af1e011b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65bcd2a87f774ec9bd3d1f66c58eda08'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:38.758447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 990/1962] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:46:38 2022] DELETE /identity/v3/users/65bcd2a87f774ec9bd3d1f66c58eda08 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:46:38.928675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:38.930097 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:38.943041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d350ec0-7fd7-47eb-8326-b7d64e6f7440 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:39.004537 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:39.005287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.005570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.005847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] PATH_INFO: `/v3/projects/360cea05ebfc4d95b3b37a53f9673b3c` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.006997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=360cea05ebfc4d95b3b37a53f9673b3c)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:39.008959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:39.015485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:39.055200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31fbec-9a60-4f3a-b121-c8390c03f0ee None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.055713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31fbec-9a60-4f3a-b121-c8390c03f0ee None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.055846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31fbec-9a60-4f3a-b121-c8390c03f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.057481 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '360cea05ebfc4d95b3b37a53f9673b3c', 'request_id': 'req-2afa903c-a571-4495-b6a3-2872d1b49e41'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:39.064982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '360cea05ebfc4d95b3b37a53f9673b3c', 'request_id': 'req-2afa903c-a571-4495-b6a3-2872d1b49e41'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:39.072124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '360cea05ebfc4d95b3b37a53f9673b3c', 'request_id': 'req-2afa903c-a571-4495-b6a3-2872d1b49e41'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:39.075880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-2b31fbec-9a60-4f3a-b121-c8390c03f0ee None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:39.077130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2b31fbec-9a60-4f3a-b121-c8390c03f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:39.083327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] The token cache is being invalidate because project 360cea05ebfc4d95b3b37a53f9673b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:46:39.083623 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:39.084259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2afa903c-a571-4495-b6a3-2872d1b49e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:46:39.085781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 973/1963] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:46:38 2022] DELETE /identity/v3/projects/360cea05ebfc4d95b3b37a53f9673b3c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:46:39.090602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab7b0a-6e5b-4ef5-b3c7-1d1d9ed97709 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.091038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab7b0a-6e5b-4ef5-b3c7-1d1d9ed97709 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.091359 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab7b0a-6e5b-4ef5-b3c7-1d1d9ed97709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.111202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d8ab7b0a-6e5b-4ef5-b3c7-1d1d9ed97709 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:39.112470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d8ab7b0a-6e5b-4ef5-b3c7-1d1d9ed97709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:39.133708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 991/1964] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:46:39.146375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31fbec-9a60-4f3a-b121-c8390c03f0ee None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:39.148637 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2b31fbec-9a60-4f3a-b121-c8390c03f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:39.177635 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 974/1965] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:46:39.185638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfe4a2-d6df-4aa6-b87e-9a817df36ecc None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.186033 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfe4a2-d6df-4aa6-b87e-9a817df36ecc None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.186407 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfe4a2-d6df-4aa6-b87e-9a817df36ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.211360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-e0cfe4a2-d6df-4aa6-b87e-9a817df36ecc None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:39.212636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e0cfe4a2-d6df-4aa6-b87e-9a817df36ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:39.224405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 975/1966] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:46:39.226108 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfe4a2-d6df-4aa6-b87e-9a817df36ecc None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:39.231391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31fbec-9a60-4f3a-b121-c8390c03f0ee None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:39.236147 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfe4a2-d6df-4aa6-b87e-9a817df36ecc None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:39.237873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9ac56-a875-413f-bd0c-a7ae84b800d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:39.238562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9ac56-a875-413f-bd0c-a7ae84b800d1 service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.239332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9ac56-a875-413f-bd0c-a7ae84b800d1 service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.239332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9ac56-a875-413f-bd0c-a7ae84b800d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.242924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f1181-b858-4bab-83bc-a187efc925d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:39.243714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f1181-b858-4bab-83bc-a187efc925d4 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.244150 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f1181-b858-4bab-83bc-a187efc925d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.244611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f1181-b858-4bab-83bc-a187efc925d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.245521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f1181-b858-4bab-83bc-a187efc925d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:46:39.246168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f1181-b858-4bab-83bc-a187efc925d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:39.248837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f1181-b858-4bab-83bc-a187efc925d4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:39.250248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9ac56-a875-413f-bd0c-a7ae84b800d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:39.251840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9ac56-a875-413f-bd0c-a7ae84b800d1 service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:39.260279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 992/1967] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:46:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 14:46:39.268380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 976/1968] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:46:39.271830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9ac56-a875-413f-bd0c-a7ae84b800d1 service swift] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:39.282413 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9ac56-a875-413f-bd0c-a7ae84b800d1 service swift] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:39.287832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7acaa-9d2b-4ac1-a66b-04279476e454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:39.288562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7acaa-9d2b-4ac1-a66b-04279476e454 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.288994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7acaa-9d2b-4ac1-a66b-04279476e454 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.289870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7acaa-9d2b-4ac1-a66b-04279476e454 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.290109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7acaa-9d2b-4ac1-a66b-04279476e454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:39.292634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7acaa-9d2b-4ac1-a66b-04279476e454 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:39.305453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:39.332391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 993/1969] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:46:39 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:46:39.333593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f1181-b858-4bab-83bc-a187efc925d4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:39.344986 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f1181-b858-4bab-83bc-a187efc925d4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:39.424777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d55c4-bc63-4b93-95ae-45769cf0196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:39.427110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d55c4-bc63-4b93-95ae-45769cf0196e admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.427766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d55c4-bc63-4b93-95ae-45769cf0196e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.428676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d55c4-bc63-4b93-95ae-45769cf0196e admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.430038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d55c4-bc63-4b93-95ae-45769cf0196e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:39.433220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d55c4-bc63-4b93-95ae-45769cf0196e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:39.480058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 977/1970] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:46:39 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:46:39.484615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7acaa-9d2b-4ac1-a66b-04279476e454 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:39.505564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7acaa-9d2b-4ac1-a66b-04279476e454 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:39.510960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f59b8-9b8c-415e-8905-4ffcf67c3568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:39.511807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f59b8-9b8c-415e-8905-4ffcf67c3568 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.512178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f59b8-9b8c-415e-8905-4ffcf67c3568 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.512648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f59b8-9b8c-415e-8905-4ffcf67c3568 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.513942 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f59b8-9b8c-415e-8905-4ffcf67c3568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:39.515762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f59b8-9b8c-415e-8905-4ffcf67c3568 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:39.531335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 994/1971] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:46:39.535245 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d55c4-bc63-4b93-95ae-45769cf0196e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:39.542160 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:39.548692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d55c4-bc63-4b93-95ae-45769cf0196e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:39.555667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f2f90-b9ba-4de4-ac7b-33323ccbdfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:39.556575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f2f90-b9ba-4de4-ac7b-33323ccbdfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.557195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f2f90-b9ba-4de4-ac7b-33323ccbdfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.557813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f2f90-b9ba-4de4-ac7b-33323ccbdfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.558592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f2f90-b9ba-4de4-ac7b-33323ccbdfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:39.561046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f2f90-b9ba-4de4-ac7b-33323ccbdfd0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:39.577257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f59b8-9b8c-415e-8905-4ffcf67c3568 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:39.583682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 978/1972] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:46:39.587740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f59b8-9b8c-415e-8905-4ffcf67c3568 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:39.592955 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097a077-f48a-451c-8038-fceb02eb6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:39.593624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097a077-f48a-451c-8038-fceb02eb6419 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.593904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097a077-f48a-451c-8038-fceb02eb6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.600287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097a077-f48a-451c-8038-fceb02eb6419 admin admin] PATH_INFO: `/v3/projects/370ff2b623ce4dfc992a266bd41ee51e/users/d7383d936b3544ccb9093a3beeb5cbd3/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.627008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097a077-f48a-451c-8038-fceb02eb6419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370ff2b623ce4dfc992a266bd41ee51e, user_id=d7383d936b3544ccb9093a3beeb5cbd3, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:39.630676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097a077-f48a-451c-8038-fceb02eb6419 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:39.654390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 995/1973] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:46:39 2022] PUT /identity/v3/projects/370ff2b623ce4dfc992a266bd41ee51e/users/d7383d936b3544ccb9093a3beeb5cbd3/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:46:39.657765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95c433-5317-439e-9354-784e9da2ec80 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.658376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95c433-5317-439e-9354-784e9da2ec80 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.658858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95c433-5317-439e-9354-784e9da2ec80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.680374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6b95c433-5317-439e-9354-784e9da2ec80 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:39.681651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6b95c433-5317-439e-9354-784e9da2ec80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:39.748719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 979/1974] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:46:39.758546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097a077-f48a-451c-8038-fceb02eb6419 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:39.760273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1097a077-f48a-451c-8038-fceb02eb6419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:39.823174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097a077-f48a-451c-8038-fceb02eb6419 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:39.826946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080699-8533-4f80-91cd-c118f84414e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:39.827557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080699-8533-4f80-91cd-c118f84414e4 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:39.827845 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080699-8533-4f80-91cd-c118f84414e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:39.828115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080699-8533-4f80-91cd-c118f84414e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:39.837640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080699-8533-4f80-91cd-c118f84414e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:39.839089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080699-8533-4f80-91cd-c118f84414e4 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:39.849427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 996/1975] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:44.995719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025268d-39c3-4e0b-ae51-f7f7b0f0d674 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:44.996317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025268d-39c3-4e0b-ae51-f7f7b0f0d674 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:44.996459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025268d-39c3-4e0b-ae51-f7f7b0f0d674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:45.054147 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-2025268d-39c3-4e0b-ae51-f7f7b0f0d674 None None] MFA Rules not processed for user `d7383d936b3544ccb9093a3beeb5cbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:45.056143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-2025268d-39c3-4e0b-ae51-f7f7b0f0d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:45.177736 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 980/1976] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:46:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:46:45.182206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080699-8533-4f80-91cd-c118f84414e4 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:45.220078 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080699-8533-4f80-91cd-c118f84414e4 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:45.225815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3ea1e-9dec-4d46-a464-6d90b4b879db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:45.226447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3ea1e-9dec-4d46-a464-6d90b4b879db admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:45.227044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3ea1e-9dec-4d46-a464-6d90b4b879db admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:45.227333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3ea1e-9dec-4d46-a464-6d90b4b879db admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:45.228060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3ea1e-9dec-4d46-a464-6d90b4b879db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:45.230015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3ea1e-9dec-4d46-a464-6d90b4b879db admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:45.276040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 997/1977] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:46:45 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:46:45.277265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025268d-39c3-4e0b-ae51-f7f7b0f0d674 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:45.291128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025268d-39c3-4e0b-ae51-f7f7b0f0d674 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:45.378259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4ca35-f76b-4fa1-928a-d0bcf4c10b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:45.379032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4ca35-f76b-4fa1-928a-d0bcf4c10b58 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:45.379493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4ca35-f76b-4fa1-928a-d0bcf4c10b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:45.379833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4ca35-f76b-4fa1-928a-d0bcf4c10b58 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:45.380668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4ca35-f76b-4fa1-928a-d0bcf4c10b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:45.384009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4ca35-f76b-4fa1-928a-d0bcf4c10b58 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:45.410890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 981/1978] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:46:45 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:46:45.414582 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3ea1e-9dec-4d46-a464-6d90b4b879db admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:45.425874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3ea1e-9dec-4d46-a464-6d90b4b879db admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:45.431201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3910f3-ea33-468c-88e0-71fb65c8ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:45.432120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3910f3-ea33-468c-88e0-71fb65c8ad84 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:45.432558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3910f3-ea33-468c-88e0-71fb65c8ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:45.432850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3910f3-ea33-468c-88e0-71fb65c8ad84 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:45.433596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3910f3-ea33-468c-88e0-71fb65c8ad84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:45.435986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3910f3-ea33-468c-88e0-71fb65c8ad84 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:45.448416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 998/1979] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:46:45.453688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4ca35-f76b-4fa1-928a-d0bcf4c10b58 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:45.473236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4ca35-f76b-4fa1-928a-d0bcf4c10b58 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:45.477927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c906b-6188-4dce-a47e-6b945dadd155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:45.478601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c906b-6188-4dce-a47e-6b945dadd155 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:45.478879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c906b-6188-4dce-a47e-6b945dadd155 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:45.479173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c906b-6188-4dce-a47e-6b945dadd155 admin admin] PATH_INFO: `/v3/projects/3c71e87c7020403982945a2019de5bc5/users/00841c795d1b4f0e9515e8b725245cee/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:45.494708 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:45.505068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c906b-6188-4dce-a47e-6b945dadd155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c71e87c7020403982945a2019de5bc5, user_id=00841c795d1b4f0e9515e8b725245cee, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:45.508556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c906b-6188-4dce-a47e-6b945dadd155 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:45.536130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 982/1980] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:46:45 2022] PUT /identity/v3/projects/3c71e87c7020403982945a2019de5bc5/users/00841c795d1b4f0e9515e8b725245cee/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:46:45.537317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3910f3-ea33-468c-88e0-71fb65c8ad84 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:45.569767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3910f3-ea33-468c-88e0-71fb65c8ad84 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:45.671526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd732470-0ed6-4b1c-9fe4-23f23f7b77ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:45.672139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd732470-0ed6-4b1c-9fe4-23f23f7b77ff admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:45.672455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd732470-0ed6-4b1c-9fe4-23f23f7b77ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:45.672728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd732470-0ed6-4b1c-9fe4-23f23f7b77ff admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:45.673370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd732470-0ed6-4b1c-9fe4-23f23f7b77ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:45.675217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd732470-0ed6-4b1c-9fe4-23f23f7b77ff admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:45.691272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 999/1981] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:46:45.692431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c906b-6188-4dce-a47e-6b945dadd155 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:45.706789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c906b-6188-4dce-a47e-6b945dadd155 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:45.711789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dfbac-cae8-4012-a4cf-4a9a4baec4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:45.712498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dfbac-cae8-4012-a4cf-4a9a4baec4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:45.712816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dfbac-cae8-4012-a4cf-4a9a4baec4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:45.713049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dfbac-cae8-4012-a4cf-4a9a4baec4d4 admin admin] PATH_INFO: `/v3/projects/3c71e87c7020403982945a2019de5bc5/users/00841c795d1b4f0e9515e8b725245cee/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:45.715196 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875dfbac-cae8-4012-a4cf-4a9a4baec4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c71e87c7020403982945a2019de5bc5, user_id=00841c795d1b4f0e9515e8b725245cee, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:45.718344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875dfbac-cae8-4012-a4cf-4a9a4baec4d4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:45.731114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 983/1982] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:46:45 2022] PUT /identity/v3/projects/3c71e87c7020403982945a2019de5bc5/users/00841c795d1b4f0e9515e8b725245cee/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:46:51.645101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c97b1f-2bc6-4fc5-b423-85ff1559ee1e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:51.645724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c97b1f-2bc6-4fc5-b423-85ff1559ee1e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:51.645724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c97b1f-2bc6-4fc5-b423-85ff1559ee1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:51.681743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-72c97b1f-2bc6-4fc5-b423-85ff1559ee1e None None] MFA Rules not processed for user `00841c795d1b4f0e9515e8b725245cee`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:51.683067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-72c97b1f-2bc6-4fc5-b423-85ff1559ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:51.693858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:51.758479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1000/1983] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:46:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:46:51.772251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dfbac-cae8-4012-a4cf-4a9a4baec4d4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:51.774118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-875dfbac-cae8-4012-a4cf-4a9a4baec4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:51.853444 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dfbac-cae8-4012-a4cf-4a9a4baec4d4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:51.860390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e1fec-fab5-4504-9ecc-8d0e3f2eae8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:51.861032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e1fec-fab5-4504-9ecc-8d0e3f2eae8d service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:51.861317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e1fec-fab5-4504-9ecc-8d0e3f2eae8d service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:51.861596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e1fec-fab5-4504-9ecc-8d0e3f2eae8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:51.921024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e1fec-fab5-4504-9ecc-8d0e3f2eae8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:51.922701 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e1fec-fab5-4504-9ecc-8d0e3f2eae8d service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:51.932621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 984/1984] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:53.022304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a8a9f-191c-4310-af8d-42f69160ca58 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:53.022792 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a8a9f-191c-4310-af8d-42f69160ca58 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:53.023384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a8a9f-191c-4310-af8d-42f69160ca58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:53.047324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-2c0a8a9f-191c-4310-af8d-42f69160ca58 None None] MFA Rules not processed for user `d7383d936b3544ccb9093a3beeb5cbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:53.048621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2c0a8a9f-191c-4310-af8d-42f69160ca58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:53.056288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1001/1985] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:46:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:46:54.495425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e1fec-fab5-4504-9ecc-8d0e3f2eae8d service nova] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:54.497120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e96e1fec-fab5-4504-9ecc-8d0e3f2eae8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:54.512717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e1fec-fab5-4504-9ecc-8d0e3f2eae8d service nova] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:54.524767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cb69d-d3be-408b-8a49-49d5e23adef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:54.525822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cb69d-d3be-408b-8a49-49d5e23adef8 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:54.526338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cb69d-d3be-408b-8a49-49d5e23adef8 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:54.526829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cb69d-d3be-408b-8a49-49d5e23adef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:54.530507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-230cb69d-d3be-408b-8a49-49d5e23adef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:54.616781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cb69d-d3be-408b-8a49-49d5e23adef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:54.618316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cb69d-d3be-408b-8a49-49d5e23adef8 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:54.627880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 985/1986] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:57.372311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a8a9f-191c-4310-af8d-42f69160ca58 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:57.388396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a8a9f-191c-4310-af8d-42f69160ca58 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:57.395953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acd548-3ea5-4f78-b1b5-9f32a1cfb0a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:57.396775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acd548-3ea5-4f78-b1b5-9f32a1cfb0a6 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:57.397161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acd548-3ea5-4f78-b1b5-9f32a1cfb0a6 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:57.397691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acd548-3ea5-4f78-b1b5-9f32a1cfb0a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:57.400105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d8acd548-3ea5-4f78-b1b5-9f32a1cfb0a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:57.448065 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acd548-3ea5-4f78-b1b5-9f32a1cfb0a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:57.450816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acd548-3ea5-4f78-b1b5-9f32a1cfb0a6 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:57.465225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1002/1987] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:46:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:46:58.287643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ebcfe-2ea1-4754-af79-d9162c3f65bb None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:58.288141 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ebcfe-2ea1-4754-af79-d9162c3f65bb None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:58.288249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ebcfe-2ea1-4754-af79-d9162c3f65bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:58.314235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-018ebcfe-2ea1-4754-af79-d9162c3f65bb None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:46:58.315616 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-018ebcfe-2ea1-4754-af79-d9162c3f65bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:46:58.321605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:46:58.393687 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 986/1988] 10.222.0.13 () {58 vars in 967 bytes} [Thu Oct 20 14:46:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:46:58.410139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acd548-3ea5-4f78-b1b5-9f32a1cfb0a6 service nova] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:46:58.479125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acd548-3ea5-4f78-b1b5-9f32a1cfb0a6 service nova] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:46:58.486240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cb6dd-4007-417a-85c2-b5e044b00bde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:46:58.487342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cb6dd-4007-417a-85c2-b5e044b00bde service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:46:58.488393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cb6dd-4007-417a-85c2-b5e044b00bde service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:46:58.488964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cb6dd-4007-417a-85c2-b5e044b00bde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:46:58.505764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cb6dd-4007-417a-85c2-b5e044b00bde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:46:58.507358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cb6dd-4007-417a-85c2-b5e044b00bde service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:46:58.530831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1003/1989] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:46:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:47:08.385593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=20 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:08.515383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=65 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:11.616710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ebcfe-2ea1-4754-af79-d9162c3f65bb None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:11.618222 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-018ebcfe-2ea1-4754-af79-d9162c3f65bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:11.676395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ebcfe-2ea1-4754-af79-d9162c3f65bb None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:11.680372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:11.681070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:11.681351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:11.681627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:11.683572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:11.694525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:11.695960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:11.704979 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 987/1990] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:17.409352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cb6dd-4007-417a-85c2-b5e044b00bde service swift] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:17.410898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f91cb6dd-4007-417a-85c2-b5e044b00bde service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:17.481136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:17.482651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:17.502816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cb6dd-4007-417a-85c2-b5e044b00bde service swift] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:17.507899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a9b31-3841-45ac-af11-b633dcf13a9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc18a78229064e8d8ea4321c3c23c4ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:17.508600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a9b31-3841-45ac-af11-b633dcf13a9b service placement] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:17.508881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a9b31-3841-45ac-af11-b633dcf13a9b service placement] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:17.509155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a9b31-3841-45ac-af11-b633dcf13a9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:17.510973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-777a9b31-3841-45ac-af11-b633dcf13a9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:17.523409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a9b31-3841-45ac-af11-b633dcf13a9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:17.525042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a9b31-3841-45ac-af11-b633dcf13a9b service placement] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:17.535305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1004/1991] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:17.553257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f4da5-3754-467f-a0b0-b418b1548a98 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:17.558193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:17.558852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:17.559128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:17.559396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] PATH_INFO: `/v3/users/ae94a3ece5f644c6893589a1df84d1c6` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:17.560622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae94a3ece5f644c6893589a1df84d1c6)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:17.563217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:17.610767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae94a3ece5f644c6893589a1df84d1c6'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.617473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae94a3ece5f644c6893589a1df84d1c6'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.618129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae94a3ece5f644c6893589a1df84d1c6'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.618734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae94a3ece5f644c6893589a1df84d1c6'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.626793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6cc770cd-f0d1-47af-80ea-24c766c30e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae94a3ece5f644c6893589a1df84d1c6'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.645325 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 988/1992] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:17 2022] DELETE /identity/v3/users/ae94a3ece5f644c6893589a1df84d1c6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:17.824408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a9b31-3841-45ac-af11-b633dcf13a9b service placement] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:17.835972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a9b31-3841-45ac-af11-b633dcf13a9b service placement] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:17.899300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:17.899936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:17.900215 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:17.900521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] PATH_INFO: `/v3/projects/7688b0fbf84b4f4f9c21cb4112ee53b5` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:17.901640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7688b0fbf84b4f4f9c21cb4112ee53b5)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:17.903562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:17.959939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7688b0fbf84b4f4f9c21cb4112ee53b5', 'request_id': 'req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.966267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7688b0fbf84b4f4f9c21cb4112ee53b5', 'request_id': 'req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.974685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7688b0fbf84b4f4f9c21cb4112ee53b5', 'request_id': 'req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.986520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] The token cache is being invalidate because project 7688b0fbf84b4f4f9c21cb4112ee53b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:17.986743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.987383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6387152a-8bc4-4f4d-b0cd-b69f2fe327d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:17.988909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1005/1993] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:17 2022] DELETE /identity/v3/projects/7688b0fbf84b4f4f9c21cb4112ee53b5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:17.993833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801f2f-3568-4efa-a30d-4df90134947b None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:17.994132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801f2f-3568-4efa-a30d-4df90134947b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:17.994406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801f2f-3568-4efa-a30d-4df90134947b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.014130 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-f8801f2f-3568-4efa-a30d-4df90134947b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:18.015309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f8801f2f-3568-4efa-a30d-4df90134947b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:18.032731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:18.081472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 989/1994] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:18.090060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48479ddd-63f2-4a1e-922a-0c9dad212846 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.090437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48479ddd-63f2-4a1e-922a-0c9dad212846 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.090820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48479ddd-63f2-4a1e-922a-0c9dad212846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.113176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-48479ddd-63f2-4a1e-922a-0c9dad212846 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:18.114402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-48479ddd-63f2-4a1e-922a-0c9dad212846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:18.123077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1006/1995] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:18.127772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801f2f-3568-4efa-a30d-4df90134947b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.138278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801f2f-3568-4efa-a30d-4df90134947b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.144047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad081170-dae1-4fbe-8843-5bf49dde9f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.144777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad081170-dae1-4fbe-8843-5bf49dde9f96 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.145087 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad081170-dae1-4fbe-8843-5bf49dde9f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.145364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad081170-dae1-4fbe-8843-5bf49dde9f96 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.146158 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad081170-dae1-4fbe-8843-5bf49dde9f96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:18.146476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad081170-dae1-4fbe-8843-5bf49dde9f96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.148134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad081170-dae1-4fbe-8843-5bf49dde9f96 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.160278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 990/1996] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:47:18.164498 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48479ddd-63f2-4a1e-922a-0c9dad212846 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.173905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48479ddd-63f2-4a1e-922a-0c9dad212846 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.178675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55be55c-40b2-4769-a398-fa9e541cc5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.179271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55be55c-40b2-4769-a398-fa9e541cc5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.179545 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55be55c-40b2-4769-a398-fa9e541cc5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.179812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55be55c-40b2-4769-a398-fa9e541cc5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.180479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55be55c-40b2-4769-a398-fa9e541cc5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.182389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55be55c-40b2-4769-a398-fa9e541cc5b1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.194445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:18.212235 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1007/1997] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:18 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:18.216717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad081170-dae1-4fbe-8843-5bf49dde9f96 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.227643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad081170-dae1-4fbe-8843-5bf49dde9f96 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.286136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeda22-c659-4c8b-820a-e8b55b7730ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.286664 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeda22-c659-4c8b-820a-e8b55b7730ed admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.286941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeda22-c659-4c8b-820a-e8b55b7730ed admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.287404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeda22-c659-4c8b-820a-e8b55b7730ed admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.288053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceeda22-c659-4c8b-820a-e8b55b7730ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.290019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceeda22-c659-4c8b-820a-e8b55b7730ed admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.312070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 991/1998] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:18 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:18.316641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55be55c-40b2-4769-a398-fa9e541cc5b1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.326884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55be55c-40b2-4769-a398-fa9e541cc5b1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.331851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a4f53-d37b-4298-8b4b-9fd6ad4ef854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.332470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a4f53-d37b-4298-8b4b-9fd6ad4ef854 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.332748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a4f53-d37b-4298-8b4b-9fd6ad4ef854 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.333018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a4f53-d37b-4298-8b4b-9fd6ad4ef854 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.333645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a4f53-d37b-4298-8b4b-9fd6ad4ef854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.335261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a4f53-d37b-4298-8b4b-9fd6ad4ef854 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.345293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1008/1999] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:18.349293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeda22-c659-4c8b-820a-e8b55b7730ed admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.361249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeda22-c659-4c8b-820a-e8b55b7730ed admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.365861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec691bd-272c-4899-b384-5bcaa7b54c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.366463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec691bd-272c-4899-b384-5bcaa7b54c1b admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.366744 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec691bd-272c-4899-b384-5bcaa7b54c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.367018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec691bd-272c-4899-b384-5bcaa7b54c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.367659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec691bd-272c-4899-b384-5bcaa7b54c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.369295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec691bd-272c-4899-b384-5bcaa7b54c1b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.380945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 992/2000] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:18.385013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a4f53-d37b-4298-8b4b-9fd6ad4ef854 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.394953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a4f53-d37b-4298-8b4b-9fd6ad4ef854 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.399628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47923773-ac42-4d69-963b-5e4a5c5beff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.400265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47923773-ac42-4d69-963b-5e4a5c5beff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.400577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47923773-ac42-4d69-963b-5e4a5c5beff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.400855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47923773-ac42-4d69-963b-5e4a5c5beff3 admin admin] PATH_INFO: `/v3/projects/9c951ccab05e478c9ac57a80fef6f6bb/users/66f852a1bcb84176b995b8f8ed538eba/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.415027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47923773-ac42-4d69-963b-5e4a5c5beff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c951ccab05e478c9ac57a80fef6f6bb, user_id=66f852a1bcb84176b995b8f8ed538eba, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.417800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47923773-ac42-4d69-963b-5e4a5c5beff3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.431135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1009/2001] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:18 2022] PUT /identity/v3/projects/9c951ccab05e478c9ac57a80fef6f6bb/users/66f852a1bcb84176b995b8f8ed538eba/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:18.441930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ff2f3-a9ca-4d8d-abe4-88e016ffc0ce None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.442243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ff2f3-a9ca-4d8d-abe4-88e016ffc0ce None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.442519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ff2f3-a9ca-4d8d-abe4-88e016ffc0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.470385 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-fe1ff2f3-a9ca-4d8d-abe4-88e016ffc0ce None None] MFA Rules not processed for user `66f852a1bcb84176b995b8f8ed538eba`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:18.471625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fe1ff2f3-a9ca-4d8d-abe4-88e016ffc0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:18.537003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 993/2002] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:18.541354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47923773-ac42-4d69-963b-5e4a5c5beff3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.606319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47923773-ac42-4d69-963b-5e4a5c5beff3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.611556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e88859-847d-47e3-91cd-9a1cf91996b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.612164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e88859-847d-47e3-91cd-9a1cf91996b0 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.612478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e88859-847d-47e3-91cd-9a1cf91996b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.612753 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e88859-847d-47e3-91cd-9a1cf91996b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.613380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e88859-847d-47e3-91cd-9a1cf91996b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.615286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e88859-847d-47e3-91cd-9a1cf91996b0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.643407 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1010/2003] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:18 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:18.647575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ff2f3-a9ca-4d8d-abe4-88e016ffc0ce None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.658933 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ff2f3-a9ca-4d8d-abe4-88e016ffc0ce None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.670980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:18.718076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b377fe-d76e-434b-b9ec-b39209a269e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.718772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b377fe-d76e-434b-b9ec-b39209a269e8 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.719077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b377fe-d76e-434b-b9ec-b39209a269e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.719457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b377fe-d76e-434b-b9ec-b39209a269e8 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.720075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b377fe-d76e-434b-b9ec-b39209a269e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.722029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b377fe-d76e-434b-b9ec-b39209a269e8 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.751968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 994/2004] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:18 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:18.753567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e88859-847d-47e3-91cd-9a1cf91996b0 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.766893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e88859-847d-47e3-91cd-9a1cf91996b0 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.771855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de7cba-14f4-44ee-815c-e70f06f5128a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.772501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de7cba-14f4-44ee-815c-e70f06f5128a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.772788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de7cba-14f4-44ee-815c-e70f06f5128a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.773061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de7cba-14f4-44ee-815c-e70f06f5128a admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.773741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de7cba-14f4-44ee-815c-e70f06f5128a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.775295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de7cba-14f4-44ee-815c-e70f06f5128a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.786322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1011/2005] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:18.790576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b377fe-d76e-434b-b9ec-b39209a269e8 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.800392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b377fe-d76e-434b-b9ec-b39209a269e8 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.804987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b7bec-5837-4d4e-b0d4-049cc6cc3964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.805633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b7bec-5837-4d4e-b0d4-049cc6cc3964 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.805907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b7bec-5837-4d4e-b0d4-049cc6cc3964 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.806176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b7bec-5837-4d4e-b0d4-049cc6cc3964 admin admin] PATH_INFO: `/v3/projects/a23a7448b8ca438e902d75ba9ac3acc9/users/5864c63964a844f49f6749d9a2135096/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.821264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b7bec-5837-4d4e-b0d4-049cc6cc3964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23a7448b8ca438e902d75ba9ac3acc9, user_id=5864c63964a844f49f6749d9a2135096, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.823914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b7bec-5837-4d4e-b0d4-049cc6cc3964 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.837741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 995/2006] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:18 2022] PUT /identity/v3/projects/a23a7448b8ca438e902d75ba9ac3acc9/users/5864c63964a844f49f6749d9a2135096/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:18.841519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de7cba-14f4-44ee-815c-e70f06f5128a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.852021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de7cba-14f4-44ee-815c-e70f06f5128a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.910481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbc799-c707-4257-8f79-a9e77435118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.911096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbc799-c707-4257-8f79-a9e77435118b admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.911384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbc799-c707-4257-8f79-a9e77435118b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.911659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbc799-c707-4257-8f79-a9e77435118b admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.912300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbc799-c707-4257-8f79-a9e77435118b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.913918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbc799-c707-4257-8f79-a9e77435118b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.928284 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1012/2007] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:18.932462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b7bec-5837-4d4e-b0d4-049cc6cc3964 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:18.942941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b7bec-5837-4d4e-b0d4-049cc6cc3964 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:18.947575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292894d1-5a0c-455b-8e28-46bbb1fc4785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:18.948230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292894d1-5a0c-455b-8e28-46bbb1fc4785 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.948575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292894d1-5a0c-455b-8e28-46bbb1fc4785 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.948857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292894d1-5a0c-455b-8e28-46bbb1fc4785 admin admin] PATH_INFO: `/v3/projects/a23a7448b8ca438e902d75ba9ac3acc9/users/5864c63964a844f49f6749d9a2135096/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:18.950778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292894d1-5a0c-455b-8e28-46bbb1fc4785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23a7448b8ca438e902d75ba9ac3acc9, user_id=5864c63964a844f49f6749d9a2135096, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:18.954106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292894d1-5a0c-455b-8e28-46bbb1fc4785 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:18.964090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 996/2008] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:18 2022] PUT /identity/v3/projects/a23a7448b8ca438e902d75ba9ac3acc9/users/5864c63964a844f49f6749d9a2135096/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:18.974925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeec3e-e0fb-46db-a3f6-99d6510fb4fb None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:18.975210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeec3e-e0fb-46db-a3f6-99d6510fb4fb None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:18.975482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeec3e-e0fb-46db-a3f6-99d6510fb4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:19.003623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f8eeec3e-e0fb-46db-a3f6-99d6510fb4fb None None] MFA Rules not processed for user `5864c63964a844f49f6749d9a2135096`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:19.004944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f8eeec3e-e0fb-46db-a3f6-99d6510fb4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:19.017568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:19.077839 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1013/2009] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:19.089579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292894d1-5a0c-455b-8e28-46bbb1fc4785 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:19.091372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-292894d1-5a0c-455b-8e28-46bbb1fc4785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:19.155831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292894d1-5a0c-455b-8e28-46bbb1fc4785 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:19.160012 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c66506-4c31-465e-8070-e22d63304692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:19.160653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c66506-4c31-465e-8070-e22d63304692 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:19.160928 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c66506-4c31-465e-8070-e22d63304692 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:19.161197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c66506-4c31-465e-8070-e22d63304692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:19.170052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c66506-4c31-465e-8070-e22d63304692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:19.171473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c66506-4c31-465e-8070-e22d63304692 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:19.180834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 997/2010] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:19.252771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeec3e-e0fb-46db-a3f6-99d6510fb4fb None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:19.365328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeec3e-e0fb-46db-a3f6-99d6510fb4fb None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:19.370191 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:19.371038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:19.371164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:19.371390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] PATH_INFO: `/v3/users/66f852a1bcb84176b995b8f8ed538eba` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:19.372617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f852a1bcb84176b995b8f8ed538eba)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:19.374466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:19.426595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f852a1bcb84176b995b8f8ed538eba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:19.433430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f852a1bcb84176b995b8f8ed538eba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:19.434101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f852a1bcb84176b995b8f8ed538eba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:19.434705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f852a1bcb84176b995b8f8ed538eba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:19.442744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f852a1bcb84176b995b8f8ed538eba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:19.460891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1014/2011] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:19 2022] DELETE /identity/v3/users/66f852a1bcb84176b995b8f8ed538eba => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:19.465436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfe619-4cea-4af6-8b56-1e1439cdd67e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:19.465734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfe619-4cea-4af6-8b56-1e1439cdd67e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:19.466016 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfe619-4cea-4af6-8b56-1e1439cdd67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:19.484986 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-8fbfe619-4cea-4af6-8b56-1e1439cdd67e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:19.486184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8fbfe619-4cea-4af6-8b56-1e1439cdd67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:19.544454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 998/2012] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:19.554578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:19.556033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:19.589691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:19.616239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a6e9-e74d-45b1-ab65-99b6ba9f1c86 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:19.619877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508111ee-f496-4f27-ab20-d2e44065c8e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:19.620514 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508111ee-f496-4f27-ab20-d2e44065c8e2 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:19.620792 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508111ee-f496-4f27-ab20-d2e44065c8e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:19.621061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508111ee-f496-4f27-ab20-d2e44065c8e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:19.630618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508111ee-f496-4f27-ab20-d2e44065c8e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:19.632039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508111ee-f496-4f27-ab20-d2e44065c8e2 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:19.641518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1015/2013] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:19.918973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfe619-4cea-4af6-8b56-1e1439cdd67e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:19.920548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8fbfe619-4cea-4af6-8b56-1e1439cdd67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:19.934648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfe619-4cea-4af6-8b56-1e1439cdd67e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:19.940092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:19.941489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:19.941784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:19.942061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] PATH_INFO: `/v3/projects/9c951ccab05e478c9ac57a80fef6f6bb` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:19.943200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c951ccab05e478c9ac57a80fef6f6bb)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:19.945159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:19.953961 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:19.993085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c951ccab05e478c9ac57a80fef6f6bb', 'request_id': 'req-c134767f-8711-4750-984a-dcdadac136f5'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:19.998572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c951ccab05e478c9ac57a80fef6f6bb', 'request_id': 'req-c134767f-8711-4750-984a-dcdadac136f5'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.006229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c951ccab05e478c9ac57a80fef6f6bb', 'request_id': 'req-c134767f-8711-4750-984a-dcdadac136f5'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.018121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] The token cache is being invalidate because project 9c951ccab05e478c9ac57a80fef6f6bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:20.018403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.019018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.020512 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 999/2014] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:19 2022] DELETE /identity/v3/projects/9c951ccab05e478c9ac57a80fef6f6bb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:20.024475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508111ee-f496-4f27-ab20-d2e44065c8e2 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:20.034291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508111ee-f496-4f27-ab20-d2e44065c8e2 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:20.089913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:20.090541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:20.090822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:20.091097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] PATH_INFO: `/v3/users/5864c63964a844f49f6749d9a2135096` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:20.092262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5864c63964a844f49f6749d9a2135096)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:20.094974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:20.140425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5864c63964a844f49f6749d9a2135096'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.147278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5864c63964a844f49f6749d9a2135096'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.147954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5864c63964a844f49f6749d9a2135096'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.148607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5864c63964a844f49f6749d9a2135096'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.155938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c81e577b-32db-4a0f-8622-bfc319f1bf3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5864c63964a844f49f6749d9a2135096'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.174528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1016/2015] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:20 2022] DELETE /identity/v3/users/5864c63964a844f49f6749d9a2135096 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:20.675030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:20.676525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:20.691811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134767f-8711-4750-984a-dcdadac136f5 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:20.748923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:20.749551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:20.749824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:20.750116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] PATH_INFO: `/v3/projects/a23a7448b8ca438e902d75ba9ac3acc9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:20.751739 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23a7448b8ca438e902d75ba9ac3acc9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:20.753666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:20.798283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23a7448b8ca438e902d75ba9ac3acc9', 'request_id': 'req-e0d4689d-0635-432a-9012-6dea2787eaff'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.803654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23a7448b8ca438e902d75ba9ac3acc9', 'request_id': 'req-e0d4689d-0635-432a-9012-6dea2787eaff'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.811018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23a7448b8ca438e902d75ba9ac3acc9', 'request_id': 'req-e0d4689d-0635-432a-9012-6dea2787eaff'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.823034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] The token cache is being invalidate because project a23a7448b8ca438e902d75ba9ac3acc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:20.823336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.824087 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e0d4689d-0635-432a-9012-6dea2787eaff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:20.825592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1000/2016] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:20 2022] DELETE /identity/v3/projects/a23a7448b8ca438e902d75ba9ac3acc9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:20.831031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91ced5-b130-4b54-b15d-415dec065c8b None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:20.831488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91ced5-b130-4b54-b15d-415dec065c8b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:20.831578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91ced5-b130-4b54-b15d-415dec065c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:20.851478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-bf91ced5-b130-4b54-b15d-415dec065c8b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:20.852685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bf91ced5-b130-4b54-b15d-415dec065c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:20.898885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:20.912750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1017/2017] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:20.921740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01a719-9586-49ba-a002-b84236be5406 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:20.921984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01a719-9586-49ba-a002-b84236be5406 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:20.922340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01a719-9586-49ba-a002-b84236be5406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:20.942136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ea01a719-9586-49ba-a002-b84236be5406 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:20.943323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ea01a719-9586-49ba-a002-b84236be5406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:20.951197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1001/2018] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:20.956081 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91ced5-b130-4b54-b15d-415dec065c8b None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:20.965535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91ced5-b130-4b54-b15d-415dec065c8b None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:20.970234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be633a-e004-4d88-b75f-b0f899c7284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:20.970822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be633a-e004-4d88-b75f-b0f899c7284a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:20.971097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be633a-e004-4d88-b75f-b0f899c7284a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:20.971365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be633a-e004-4d88-b75f-b0f899c7284a admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:20.972002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be633a-e004-4d88-b75f-b0f899c7284a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:20.972309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be633a-e004-4d88-b75f-b0f899c7284a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:20.973917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be633a-e004-4d88-b75f-b0f899c7284a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:20.985612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1018/2019] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:47:20.989651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01a719-9586-49ba-a002-b84236be5406 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:20.998643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01a719-9586-49ba-a002-b84236be5406 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.003886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105db7d-9efa-4201-916f-c451981660c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.004507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105db7d-9efa-4201-916f-c451981660c6 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.004786 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105db7d-9efa-4201-916f-c451981660c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.005058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105db7d-9efa-4201-916f-c451981660c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.005697 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105db7d-9efa-4201-916f-c451981660c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.007655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105db7d-9efa-4201-916f-c451981660c6 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.028726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:21.039667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1002/2020] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:20 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:21.041279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be633a-e004-4d88-b75f-b0f899c7284a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.051514 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be633a-e004-4d88-b75f-b0f899c7284a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.108576 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1931e-6a48-479b-8460-fa8f4b0a403a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.109182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1931e-6a48-479b-8460-fa8f4b0a403a admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.109465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1931e-6a48-479b-8460-fa8f4b0a403a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.110335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1931e-6a48-479b-8460-fa8f4b0a403a admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.110969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1931e-6a48-479b-8460-fa8f4b0a403a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.112907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1931e-6a48-479b-8460-fa8f4b0a403a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.133818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1019/2021] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:21 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:21.137973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105db7d-9efa-4201-916f-c451981660c6 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.148038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105db7d-9efa-4201-916f-c451981660c6 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.152667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065fcd4-9626-47a7-bb48-759ca6adbbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.153258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065fcd4-9626-47a7-bb48-759ca6adbbac admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.153535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065fcd4-9626-47a7-bb48-759ca6adbbac admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.153805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065fcd4-9626-47a7-bb48-759ca6adbbac admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.154438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065fcd4-9626-47a7-bb48-759ca6adbbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.156013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065fcd4-9626-47a7-bb48-759ca6adbbac admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.165710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1003/2022] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:21.169559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1931e-6a48-479b-8460-fa8f4b0a403a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.179841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1931e-6a48-479b-8460-fa8f4b0a403a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.184373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edfad6-2792-4f11-a608-6eff540f7d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.184990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edfad6-2792-4f11-a608-6eff540f7d9c admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.185252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edfad6-2792-4f11-a608-6eff540f7d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.185529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edfad6-2792-4f11-a608-6eff540f7d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.186163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edfad6-2792-4f11-a608-6eff540f7d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.188401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edfad6-2792-4f11-a608-6eff540f7d9c admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.198621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1020/2023] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:21.202549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065fcd4-9626-47a7-bb48-759ca6adbbac admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.212562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065fcd4-9626-47a7-bb48-759ca6adbbac admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.217228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a93158-ac17-40f8-b97b-fe1a1420dcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.217873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a93158-ac17-40f8-b97b-fe1a1420dcbc admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.218153 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a93158-ac17-40f8-b97b-fe1a1420dcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.218429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a93158-ac17-40f8-b97b-fe1a1420dcbc admin admin] PATH_INFO: `/v3/projects/e402b4f743ff49bdb98bd82c7f29ca90/users/e89ca779fb0e448eb0dc0b510647e849/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.233205 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a93158-ac17-40f8-b97b-fe1a1420dcbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e402b4f743ff49bdb98bd82c7f29ca90, user_id=e89ca779fb0e448eb0dc0b510647e849, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.236468 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a93158-ac17-40f8-b97b-fe1a1420dcbc admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.251071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1004/2024] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:21 2022] PUT /identity/v3/projects/e402b4f743ff49bdb98bd82c7f29ca90/users/e89ca779fb0e448eb0dc0b510647e849/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:21.261075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3799918-814c-484c-841f-e3316bc7eccf None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.261411 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3799918-814c-484c-841f-e3316bc7eccf None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.261688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3799918-814c-484c-841f-e3316bc7eccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.288651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c3799918-814c-484c-841f-e3316bc7eccf None None] MFA Rules not processed for user `e89ca779fb0e448eb0dc0b510647e849`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:21.289857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c3799918-814c-484c-841f-e3316bc7eccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:21.355665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1021/2025] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:21.361171 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a93158-ac17-40f8-b97b-fe1a1420dcbc admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.421788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a93158-ac17-40f8-b97b-fe1a1420dcbc admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.427445 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83da03-70aa-44c9-b021-d27ec2dd0d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.428054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83da03-70aa-44c9-b021-d27ec2dd0d92 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.428383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83da03-70aa-44c9-b021-d27ec2dd0d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.428683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83da03-70aa-44c9-b021-d27ec2dd0d92 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.429313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83da03-70aa-44c9-b021-d27ec2dd0d92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.431277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83da03-70aa-44c9-b021-d27ec2dd0d92 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.461159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1005/2026] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:21 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:21.464200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3799918-814c-484c-841f-e3316bc7eccf None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.475940 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3799918-814c-484c-841f-e3316bc7eccf None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.522739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:21.536029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574c92b-76eb-410b-9d90-8359d7a90d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.536671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574c92b-76eb-410b-9d90-8359d7a90d7e admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.536953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574c92b-76eb-410b-9d90-8359d7a90d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.537223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574c92b-76eb-410b-9d90-8359d7a90d7e admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.537860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574c92b-76eb-410b-9d90-8359d7a90d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.539793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574c92b-76eb-410b-9d90-8359d7a90d7e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.560210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1022/2027] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:21 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:21.564858 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83da03-70aa-44c9-b021-d27ec2dd0d92 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.574894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83da03-70aa-44c9-b021-d27ec2dd0d92 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.580287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a06cdb-d528-4f4e-9f7e-a17d7356b990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.580919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a06cdb-d528-4f4e-9f7e-a17d7356b990 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.581198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a06cdb-d528-4f4e-9f7e-a17d7356b990 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.581470 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a06cdb-d528-4f4e-9f7e-a17d7356b990 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.582105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a06cdb-d528-4f4e-9f7e-a17d7356b990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.583691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a06cdb-d528-4f4e-9f7e-a17d7356b990 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.594186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1006/2028] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:21.597987 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574c92b-76eb-410b-9d90-8359d7a90d7e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.608206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574c92b-76eb-410b-9d90-8359d7a90d7e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.613034 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd2844-c294-4beb-a7ad-9c1b41311dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.613634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd2844-c294-4beb-a7ad-9c1b41311dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.613939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd2844-c294-4beb-a7ad-9c1b41311dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.614244 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd2844-c294-4beb-a7ad-9c1b41311dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.614889 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd2844-c294-4beb-a7ad-9c1b41311dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.616515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd2844-c294-4beb-a7ad-9c1b41311dd7 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.630281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1023/2029] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:21.631383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a06cdb-d528-4f4e-9f7e-a17d7356b990 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.642074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a06cdb-d528-4f4e-9f7e-a17d7356b990 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.646816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f362ccd-9ac9-4837-bf60-330933005652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.647477 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f362ccd-9ac9-4837-bf60-330933005652 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.647757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f362ccd-9ac9-4837-bf60-330933005652 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.648021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f362ccd-9ac9-4837-bf60-330933005652 admin admin] PATH_INFO: `/v3/projects/c32f088e14c8478c97c18bb1858a5b44/users/99bedeccf9184cc5bc1f31dedec763fe/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.664378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f362ccd-9ac9-4837-bf60-330933005652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32f088e14c8478c97c18bb1858a5b44, user_id=99bedeccf9184cc5bc1f31dedec763fe, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.667120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f362ccd-9ac9-4837-bf60-330933005652 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.681494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1007/2030] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:21 2022] PUT /identity/v3/projects/c32f088e14c8478c97c18bb1858a5b44/users/99bedeccf9184cc5bc1f31dedec763fe/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:21.693236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3eff-18ed-46de-bd05-b08587892ff1 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.693610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3eff-18ed-46de-bd05-b08587892ff1 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.693959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3eff-18ed-46de-bd05-b08587892ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.724939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-69df3eff-18ed-46de-bd05-b08587892ff1 None None] MFA Rules not processed for user `99bedeccf9184cc5bc1f31dedec763fe`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:21.726301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-69df3eff-18ed-46de-bd05-b08587892ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:21.795092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1024/2031] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:21.800505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f362ccd-9ac9-4837-bf60-330933005652 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.859831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f362ccd-9ac9-4837-bf60-330933005652 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.864460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93429f-a152-40d0-90ce-3d9245405a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.865066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93429f-a152-40d0-90ce-3d9245405a25 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.865339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93429f-a152-40d0-90ce-3d9245405a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.865608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93429f-a152-40d0-90ce-3d9245405a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.866228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93429f-a152-40d0-90ce-3d9245405a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.868124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93429f-a152-40d0-90ce-3d9245405a25 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.888798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:21.897708 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1008/2032] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:21 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:21.901305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df3eff-18ed-46de-bd05-b08587892ff1 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:21.913742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df3eff-18ed-46de-bd05-b08587892ff1 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:21.970160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dd751-a150-4b8a-9cec-dc5f4ff8583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:21.970772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dd751-a150-4b8a-9cec-dc5f4ff8583f admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:21.971052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dd751-a150-4b8a-9cec-dc5f4ff8583f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:21.971329 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dd751-a150-4b8a-9cec-dc5f4ff8583f admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:21.971956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dd751-a150-4b8a-9cec-dc5f4ff8583f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:21.973922 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dd751-a150-4b8a-9cec-dc5f4ff8583f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:21.995486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1025/2033] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:21 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:21.999559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93429f-a152-40d0-90ce-3d9245405a25 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:22.010724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93429f-a152-40d0-90ce-3d9245405a25 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:22.015269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d1cb4-0e7e-4604-97b7-3701400b7bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:22.016263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d1cb4-0e7e-4604-97b7-3701400b7bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:22.016591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d1cb4-0e7e-4604-97b7-3701400b7bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:22.016867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d1cb4-0e7e-4604-97b7-3701400b7bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:22.017508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d1cb4-0e7e-4604-97b7-3701400b7bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:22.019105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d1cb4-0e7e-4604-97b7-3701400b7bd0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:22.029387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1009/2034] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:22.033483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dd751-a150-4b8a-9cec-dc5f4ff8583f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:22.043360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dd751-a150-4b8a-9cec-dc5f4ff8583f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:22.048687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a6ce3-2019-4605-bd69-6ff9f4f59f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:22.049345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6ce3-2019-4605-bd69-6ff9f4f59f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:22.049606 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6ce3-2019-4605-bd69-6ff9f4f59f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:22.049903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6ce3-2019-4605-bd69-6ff9f4f59f62 admin admin] PATH_INFO: `/v3/projects/f4c98c1ba308479fa849aa0518395abd/users/e8fad4d81b904aa686ff13355c7a752d/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:22.065734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a6ce3-2019-4605-bd69-6ff9f4f59f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c98c1ba308479fa849aa0518395abd, user_id=e8fad4d81b904aa686ff13355c7a752d, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:22.069844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a6ce3-2019-4605-bd69-6ff9f4f59f62 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:22.082123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1026/2035] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:22 2022] PUT /identity/v3/projects/f4c98c1ba308479fa849aa0518395abd/users/e8fad4d81b904aa686ff13355c7a752d/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:22.086058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d1cb4-0e7e-4604-97b7-3701400b7bd0 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:22.095972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d1cb4-0e7e-4604-97b7-3701400b7bd0 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:22.149803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0887d-24b8-40a1-b3aa-b22bbddf4301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:22.150402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0887d-24b8-40a1-b3aa-b22bbddf4301 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:22.150679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0887d-24b8-40a1-b3aa-b22bbddf4301 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:22.150950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0887d-24b8-40a1-b3aa-b22bbddf4301 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:22.151585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0887d-24b8-40a1-b3aa-b22bbddf4301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:22.153225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0887d-24b8-40a1-b3aa-b22bbddf4301 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:22.163316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1010/2036] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:22.167475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a6ce3-2019-4605-bd69-6ff9f4f59f62 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:22.178137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a6ce3-2019-4605-bd69-6ff9f4f59f62 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:22.182778 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e407ca-a256-485c-9c12-68908dc1f301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:22.183418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e407ca-a256-485c-9c12-68908dc1f301 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:22.183697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e407ca-a256-485c-9c12-68908dc1f301 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:22.183970 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e407ca-a256-485c-9c12-68908dc1f301 admin admin] PATH_INFO: `/v3/projects/f4c98c1ba308479fa849aa0518395abd/users/e8fad4d81b904aa686ff13355c7a752d/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:22.185976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e407ca-a256-485c-9c12-68908dc1f301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c98c1ba308479fa849aa0518395abd, user_id=e8fad4d81b904aa686ff13355c7a752d, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:22.189596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e407ca-a256-485c-9c12-68908dc1f301 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:22.199531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1027/2037] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:22 2022] PUT /identity/v3/projects/f4c98c1ba308479fa849aa0518395abd/users/e8fad4d81b904aa686ff13355c7a752d/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:22.210484 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7be65-b902-431d-9e63-013b0ca6fe8e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:22.210768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7be65-b902-431d-9e63-013b0ca6fe8e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:22.211044 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7be65-b902-431d-9e63-013b0ca6fe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:22.238737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-bda7be65-b902-431d-9e63-013b0ca6fe8e None None] MFA Rules not processed for user `e8fad4d81b904aa686ff13355c7a752d`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:22.239871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bda7be65-b902-431d-9e63-013b0ca6fe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:22.307772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1011/2038] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:22.319536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e407ca-a256-485c-9c12-68908dc1f301 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:22.321254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a3e407ca-a256-485c-9c12-68908dc1f301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:22.332652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:22.382281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e407ca-a256-485c-9c12-68908dc1f301 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:22.385952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98930f-d25f-4c92-9895-b21673e6fe76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:22.386554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98930f-d25f-4c92-9895-b21673e6fe76 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:22.386836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98930f-d25f-4c92-9895-b21673e6fe76 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:22.387111 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98930f-d25f-4c92-9895-b21673e6fe76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:22.396098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98930f-d25f-4c92-9895-b21673e6fe76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:22.397562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98930f-d25f-4c92-9895-b21673e6fe76 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:22.407431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1028/2039] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:22.447760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7be65-b902-431d-9e63-013b0ca6fe8e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:22.449195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bda7be65-b902-431d-9e63-013b0ca6fe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:22.460910 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7be65-b902-431d-9e63-013b0ca6fe8e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:22.464706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfb23f-69bd-439d-b06e-35b1727be916 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:22.465295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfb23f-69bd-439d-b06e-35b1727be916 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:22.465568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfb23f-69bd-439d-b06e-35b1727be916 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:22.465836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfb23f-69bd-439d-b06e-35b1727be916 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:22.526556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfb23f-69bd-439d-b06e-35b1727be916 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:22.527994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfb23f-69bd-439d-b06e-35b1727be916 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:22.537569 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1012/2040] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:25.480164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98930f-d25f-4c92-9895-b21673e6fe76 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:25.547654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98930f-d25f-4c92-9895-b21673e6fe76 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:25.552178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:25.552843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:25.553123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:25.553400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] PATH_INFO: `/v3/users/e89ca779fb0e448eb0dc0b510647e849` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:25.554526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e89ca779fb0e448eb0dc0b510647e849)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:25.556495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:25.611004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e89ca779fb0e448eb0dc0b510647e849'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:25.617821 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e89ca779fb0e448eb0dc0b510647e849'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:25.618473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e89ca779fb0e448eb0dc0b510647e849'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:25.619072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e89ca779fb0e448eb0dc0b510647e849'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:25.626640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e89ca779fb0e448eb0dc0b510647e849'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:25.643708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1029/2041] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:25 2022] DELETE /identity/v3/users/e89ca779fb0e448eb0dc0b510647e849 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:25.648425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3e2fc-7fba-4cd6-99b6-fbfcc9ef9bf0 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:25.648901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3e2fc-7fba-4cd6-99b6-fbfcc9ef9bf0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:25.648999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3e2fc-7fba-4cd6-99b6-fbfcc9ef9bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:25.662077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:25.668565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-bbd3e2fc-7fba-4cd6-99b6-fbfcc9ef9bf0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:25.669765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bbd3e2fc-7fba-4cd6-99b6-fbfcc9ef9bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:25.733981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1013/2042] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:25.744192 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:25.745626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:25.787927 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:25.818406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56c126-8a7d-4540-9153-20bd43163ee2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:25.822259 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966bb68-e98a-472d-87d3-6730685bda3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:25.823007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966bb68-e98a-472d-87d3-6730685bda3f service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:25.823288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966bb68-e98a-472d-87d3-6730685bda3f service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:25.823884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966bb68-e98a-472d-87d3-6730685bda3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:25.833885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966bb68-e98a-472d-87d3-6730685bda3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:25.835634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966bb68-e98a-472d-87d3-6730685bda3f service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:25.846852 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1030/2043] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:26.108229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3e2fc-7fba-4cd6-99b6-fbfcc9ef9bf0 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:26.110310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bbd3e2fc-7fba-4cd6-99b6-fbfcc9ef9bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:26.122989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3e2fc-7fba-4cd6-99b6-fbfcc9ef9bf0 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:26.128675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:26.129317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:26.129598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:26.129871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] PATH_INFO: `/v3/projects/e402b4f743ff49bdb98bd82c7f29ca90` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:26.131062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e402b4f743ff49bdb98bd82c7f29ca90)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:26.133052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:26.180181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e402b4f743ff49bdb98bd82c7f29ca90', 'request_id': 'req-035413d5-3eac-4ca5-ba30-becde57d66e9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.185768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e402b4f743ff49bdb98bd82c7f29ca90', 'request_id': 'req-035413d5-3eac-4ca5-ba30-becde57d66e9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.193317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e402b4f743ff49bdb98bd82c7f29ca90', 'request_id': 'req-035413d5-3eac-4ca5-ba30-becde57d66e9'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.205119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] The token cache is being invalidate because project e402b4f743ff49bdb98bd82c7f29ca90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:26.205409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.206040 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.207716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1014/2044] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:26 2022] DELETE /identity/v3/projects/e402b4f743ff49bdb98bd82c7f29ca90 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:26.211519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966bb68-e98a-472d-87d3-6730685bda3f service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:26.221325 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966bb68-e98a-472d-87d3-6730685bda3f service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:26.286938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:26.287580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:26.287863 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:26.288143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] PATH_INFO: `/v3/users/99bedeccf9184cc5bc1f31dedec763fe` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:26.289338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] RBAC: Authorizing `identity:delete_user(user_id=99bedeccf9184cc5bc1f31dedec763fe)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:26.291271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:26.344758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99bedeccf9184cc5bc1f31dedec763fe'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.351696 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99bedeccf9184cc5bc1f31dedec763fe'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.352410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99bedeccf9184cc5bc1f31dedec763fe'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.353021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99bedeccf9184cc5bc1f31dedec763fe'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.361389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99bedeccf9184cc5bc1f31dedec763fe'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.380843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1031/2045] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:26 2022] DELETE /identity/v3/users/99bedeccf9184cc5bc1f31dedec763fe => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:26.762498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:26.763975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:26.778865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035413d5-3eac-4ca5-ba30-becde57d66e9 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:26.839839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:26.840519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:26.840796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:26.841066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] PATH_INFO: `/v3/projects/c32f088e14c8478c97c18bb1858a5b44` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:26.842170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32f088e14c8478c97c18bb1858a5b44)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:26.844688 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:26.879986 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:26.893973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32f088e14c8478c97c18bb1858a5b44', 'request_id': 'req-452f696f-21fb-418b-be17-55f85b23b984'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.899638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32f088e14c8478c97c18bb1858a5b44', 'request_id': 'req-452f696f-21fb-418b-be17-55f85b23b984'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.908129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32f088e14c8478c97c18bb1858a5b44', 'request_id': 'req-452f696f-21fb-418b-be17-55f85b23b984'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.921115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] The token cache is being invalidate because project c32f088e14c8478c97c18bb1858a5b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:26.921412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.922250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:26.924061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1015/2046] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:26 2022] DELETE /identity/v3/projects/c32f088e14c8478c97c18bb1858a5b44 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:26.928287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:26.929773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:26.943193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99840052-e09d-4ced-a0af-cb2d0487097f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:26.992458 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:27.003031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:27.003846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:27.004134 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:27.004446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] PATH_INFO: `/v3/users/e8fad4d81b904aa686ff13355c7a752d` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:27.005641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8fad4d81b904aa686ff13355c7a752d)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:27.007589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:27.076042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8fad4d81b904aa686ff13355c7a752d'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.087637 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8fad4d81b904aa686ff13355c7a752d'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.088366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8fad4d81b904aa686ff13355c7a752d'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.089072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8fad4d81b904aa686ff13355c7a752d'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.098527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e60dfa47-7c9d-40b9-9ebf-7e445f814751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8fad4d81b904aa686ff13355c7a752d'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.118619 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1032/2047] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:26 2022] DELETE /identity/v3/users/e8fad4d81b904aa686ff13355c7a752d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:27.384509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:27.386057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:27.399837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f696f-21fb-418b-be17-55f85b23b984 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:27.459407 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:27.460051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:27.460333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:27.460634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] PATH_INFO: `/v3/projects/f4c98c1ba308479fa849aa0518395abd` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:27.461884 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c98c1ba308479fa849aa0518395abd)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:27.464319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:27.511869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c98c1ba308479fa849aa0518395abd', 'request_id': 'req-4650e94c-334d-4193-a4ff-d1382761134b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.517792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c98c1ba308479fa849aa0518395abd', 'request_id': 'req-4650e94c-334d-4193-a4ff-d1382761134b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.524946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c98c1ba308479fa849aa0518395abd', 'request_id': 'req-4650e94c-334d-4193-a4ff-d1382761134b'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.537088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] The token cache is being invalidate because project f4c98c1ba308479fa849aa0518395abd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:27.537372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.538020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-4650e94c-334d-4193-a4ff-d1382761134b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:27.539516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1016/2048] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:27 2022] DELETE /identity/v3/projects/f4c98c1ba308479fa849aa0518395abd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:27.546405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb936fb-8f8f-4e2e-82fd-857eda1c31da None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:27.546809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb936fb-8f8f-4e2e-82fd-857eda1c31da None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:27.547717 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb936fb-8f8f-4e2e-82fd-857eda1c31da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:27.571930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-3bb936fb-8f8f-4e2e-82fd-857eda1c31da None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:27.573066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3bb936fb-8f8f-4e2e-82fd-857eda1c31da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:27.647332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1033/2049] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:27.656014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946bb10-f22e-4617-ba84-60dbd5bfb0ea None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:27.656909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946bb10-f22e-4617-ba84-60dbd5bfb0ea None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:27.657183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946bb10-f22e-4617-ba84-60dbd5bfb0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:27.684765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-9946bb10-f22e-4617-ba84-60dbd5bfb0ea None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:27.686071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9946bb10-f22e-4617-ba84-60dbd5bfb0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:27.694731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1017/2050] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:27.699105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb936fb-8f8f-4e2e-82fd-857eda1c31da None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:27.713857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb936fb-8f8f-4e2e-82fd-857eda1c31da None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:27.719296 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9acd5-e23e-4774-bd65-afc3935f6f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:27.719938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9acd5-e23e-4774-bd65-afc3935f6f42 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:27.720190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9acd5-e23e-4774-bd65-afc3935f6f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:27.720493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9acd5-e23e-4774-bd65-afc3935f6f42 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:27.721179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9acd5-e23e-4774-bd65-afc3935f6f42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:27.721516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9acd5-e23e-4774-bd65-afc3935f6f42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:27.723126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9acd5-e23e-4774-bd65-afc3935f6f42 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:27.740490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1034/2051] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:47:27.745424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946bb10-f22e-4617-ba84-60dbd5bfb0ea None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:27.757782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946bb10-f22e-4617-ba84-60dbd5bfb0ea None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:27.762936 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee4638-7644-41d6-9793-57c3e04ea6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:27.764391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee4638-7644-41d6-9793-57c3e04ea6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:27.764835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee4638-7644-41d6-9793-57c3e04ea6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:27.765259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee4638-7644-41d6-9793-57c3e04ea6c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:27.766393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee4638-7644-41d6-9793-57c3e04ea6c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:27.769808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee4638-7644-41d6-9793-57c3e04ea6c2 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:27.803427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1018/2052] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:27 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:27.807775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9acd5-e23e-4774-bd65-afc3935f6f42 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:27.827547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9acd5-e23e-4774-bd65-afc3935f6f42 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:27.901881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86748767-2148-49c6-8c51-bb16c0e39944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:27.902509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86748767-2148-49c6-8c51-bb16c0e39944 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:27.903475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86748767-2148-49c6-8c51-bb16c0e39944 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:27.903911 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86748767-2148-49c6-8c51-bb16c0e39944 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:27.904592 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86748767-2148-49c6-8c51-bb16c0e39944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:27.907161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86748767-2148-49c6-8c51-bb16c0e39944 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:27.915346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:27.931994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1035/2053] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:27 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:27.936222 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee4638-7644-41d6-9793-57c3e04ea6c2 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:27.947430 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee4638-7644-41d6-9793-57c3e04ea6c2 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:27.952126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879029f-e6fb-4eca-bff6-be6fa7b9cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:27.952758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879029f-e6fb-4eca-bff6-be6fa7b9cb42 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:27.953063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879029f-e6fb-4eca-bff6-be6fa7b9cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:27.953334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879029f-e6fb-4eca-bff6-be6fa7b9cb42 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:27.953974 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879029f-e6fb-4eca-bff6-be6fa7b9cb42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:27.955601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879029f-e6fb-4eca-bff6-be6fa7b9cb42 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:27.965897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1019/2054] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:27.970360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86748767-2148-49c6-8c51-bb16c0e39944 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:27.981436 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86748767-2148-49c6-8c51-bb16c0e39944 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:27.986836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24682fb3-f588-4f1b-981a-835ce4a24cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:27.987435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24682fb3-f588-4f1b-981a-835ce4a24cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:27.987713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24682fb3-f588-4f1b-981a-835ce4a24cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:27.987983 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24682fb3-f588-4f1b-981a-835ce4a24cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:27.988664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24682fb3-f588-4f1b-981a-835ce4a24cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:27.990258 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24682fb3-f588-4f1b-981a-835ce4a24cb9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:28.005171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1036/2055] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:28.009379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879029f-e6fb-4eca-bff6-be6fa7b9cb42 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:28.020177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879029f-e6fb-4eca-bff6-be6fa7b9cb42 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:28.024841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3bb3b-9b16-465f-b198-396f63c45fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:28.025498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3bb3b-9b16-465f-b198-396f63c45fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.025780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3bb3b-9b16-465f-b198-396f63c45fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.026055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3bb3b-9b16-465f-b198-396f63c45fc3 admin admin] PATH_INFO: `/v3/projects/72ee5752a7e64363a42c9d1f260b1f3d/users/adbbf48a0b014610abb533bd904da689/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.034637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:28.042399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3bb3b-9b16-465f-b198-396f63c45fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ee5752a7e64363a42c9d1f260b1f3d, user_id=adbbf48a0b014610abb533bd904da689, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:28.045751 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3bb3b-9b16-465f-b198-396f63c45fc3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:28.061730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1020/2056] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:28 2022] PUT /identity/v3/projects/72ee5752a7e64363a42c9d1f260b1f3d/users/adbbf48a0b014610abb533bd904da689/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:28.075208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab02a4-28c6-4cb0-8e39-bf6241ccc522 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.075517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab02a4-28c6-4cb0-8e39-bf6241ccc522 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.075738 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab02a4-28c6-4cb0-8e39-bf6241ccc522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.105190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-bdab02a4-28c6-4cb0-8e39-bf6241ccc522 None None] MFA Rules not processed for user `adbbf48a0b014610abb533bd904da689`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:28.106460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bdab02a4-28c6-4cb0-8e39-bf6241ccc522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:28.173961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1037/2057] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:28.178398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3bb3b-9b16-465f-b198-396f63c45fc3 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:28.243864 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3bb3b-9b16-465f-b198-396f63c45fc3 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:28.248545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0383c96-cca6-476e-b924-81d57a7a0e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:28.249259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0383c96-cca6-476e-b924-81d57a7a0e9a admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.249539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0383c96-cca6-476e-b924-81d57a7a0e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.249866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0383c96-cca6-476e-b924-81d57a7a0e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.251007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0383c96-cca6-476e-b924-81d57a7a0e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:28.252944 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0383c96-cca6-476e-b924-81d57a7a0e9a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:28.284150 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1021/2058] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:28 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:28.285300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab02a4-28c6-4cb0-8e39-bf6241ccc522 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:28.295966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab02a4-28c6-4cb0-8e39-bf6241ccc522 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:28.372547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2f64-9d5e-4463-be47-2fdc132e23c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:28.372547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2f64-9d5e-4463-be47-2fdc132e23c9 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.372547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2f64-9d5e-4463-be47-2fdc132e23c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.372547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2f64-9d5e-4463-be47-2fdc132e23c9 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.372935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d2f64-9d5e-4463-be47-2fdc132e23c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:28.372935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d2f64-9d5e-4463-be47-2fdc132e23c9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:28.396609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1038/2059] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:28 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:28.397611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0383c96-cca6-476e-b924-81d57a7a0e9a admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:28.408699 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0383c96-cca6-476e-b924-81d57a7a0e9a admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:28.413249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed548f-8b41-4042-940e-a13b56254132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:28.413851 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed548f-8b41-4042-940e-a13b56254132 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.414132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed548f-8b41-4042-940e-a13b56254132 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.414403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed548f-8b41-4042-940e-a13b56254132 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.415040 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed548f-8b41-4042-940e-a13b56254132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:28.416676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed548f-8b41-4042-940e-a13b56254132 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:28.428022 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1022/2060] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:28.432019 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2f64-9d5e-4463-be47-2fdc132e23c9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:28.442878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2f64-9d5e-4463-be47-2fdc132e23c9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:28.447392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9453a-eb61-4aaa-98c7-ec526c92eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:28.448040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9453a-eb61-4aaa-98c7-ec526c92eb13 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.448317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9453a-eb61-4aaa-98c7-ec526c92eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.448623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9453a-eb61-4aaa-98c7-ec526c92eb13 admin admin] PATH_INFO: `/v3/projects/cbbd23918e7648ebbcc13dd1e4f48b0f/users/d42774d511c947a38347f93e72c31735/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.463518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9453a-eb61-4aaa-98c7-ec526c92eb13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbd23918e7648ebbcc13dd1e4f48b0f, user_id=d42774d511c947a38347f93e72c31735, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:28.466245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9453a-eb61-4aaa-98c7-ec526c92eb13 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:28.480170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1039/2061] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:28 2022] PUT /identity/v3/projects/cbbd23918e7648ebbcc13dd1e4f48b0f/users/d42774d511c947a38347f93e72c31735/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:28.484063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed548f-8b41-4042-940e-a13b56254132 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:28.494020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed548f-8b41-4042-940e-a13b56254132 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:28.556539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc8533-66b8-4aa7-bd44-51f4efb9659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:28.557185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc8533-66b8-4aa7-bd44-51f4efb9659f admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.557605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc8533-66b8-4aa7-bd44-51f4efb9659f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.557893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc8533-66b8-4aa7-bd44-51f4efb9659f admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.558543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc8533-66b8-4aa7-bd44-51f4efb9659f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:28.560139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc8533-66b8-4aa7-bd44-51f4efb9659f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:28.574593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1023/2062] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:28.576085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9453a-eb61-4aaa-98c7-ec526c92eb13 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:28.587383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9453a-eb61-4aaa-98c7-ec526c92eb13 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:28.592061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576de6-242a-4847-840c-5390d7b63271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:28.592743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576de6-242a-4847-840c-5390d7b63271 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.593023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576de6-242a-4847-840c-5390d7b63271 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.593296 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576de6-242a-4847-840c-5390d7b63271 admin admin] PATH_INFO: `/v3/projects/cbbd23918e7648ebbcc13dd1e4f48b0f/users/d42774d511c947a38347f93e72c31735/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.595984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36576de6-242a-4847-840c-5390d7b63271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbd23918e7648ebbcc13dd1e4f48b0f, user_id=d42774d511c947a38347f93e72c31735, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:28.599309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36576de6-242a-4847-840c-5390d7b63271 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:28.610774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1040/2063] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:28 2022] PUT /identity/v3/projects/cbbd23918e7648ebbcc13dd1e4f48b0f/users/d42774d511c947a38347f93e72c31735/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:28.621100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e97143-0428-47bb-9b02-60de7502ad2b None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.621519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e97143-0428-47bb-9b02-60de7502ad2b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.621929 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e97143-0428-47bb-9b02-60de7502ad2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.650735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-30e97143-0428-47bb-9b02-60de7502ad2b None None] MFA Rules not processed for user `d42774d511c947a38347f93e72c31735`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:28.651933 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-30e97143-0428-47bb-9b02-60de7502ad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:28.698650 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:28.722836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1024/2064] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:28.734635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576de6-242a-4847-840c-5390d7b63271 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:28.736429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-36576de6-242a-4847-840c-5390d7b63271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:28.758651 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:28.799174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576de6-242a-4847-840c-5390d7b63271 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:28.802811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c82c7-a1ac-4606-b840-a29b64c20f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:28.803403 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c82c7-a1ac-4606-b840-a29b64c20f74 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:28.803676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c82c7-a1ac-4606-b840-a29b64c20f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:28.803945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c82c7-a1ac-4606-b840-a29b64c20f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:28.858635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c82c7-a1ac-4606-b840-a29b64c20f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:28.860780 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c82c7-a1ac-4606-b840-a29b64c20f74 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:28.869899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1041/2065] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:33.253000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e97143-0428-47bb-9b02-60de7502ad2b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:33.254679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-30e97143-0428-47bb-9b02-60de7502ad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:33.270280 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e97143-0428-47bb-9b02-60de7502ad2b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:33.274697 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076790e-614b-4e98-89e8-a88d34550fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:33.275043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076790e-614b-4e98-89e8-a88d34550fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:33.275419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076790e-614b-4e98-89e8-a88d34550fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:33.275702 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076790e-614b-4e98-89e8-a88d34550fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:33.288675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076790e-614b-4e98-89e8-a88d34550fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:33.290552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076790e-614b-4e98-89e8-a88d34550fc5 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:33.305121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1025/2066] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:47:37.630002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c82c7-a1ac-4606-b840-a29b64c20f74 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:37.631522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-390c82c7-a1ac-4606-b840-a29b64c20f74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:37.690493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c82c7-a1ac-4606-b840-a29b64c20f74 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:37.694862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15955f9f-1dbd-4ee8-a2e4-a382a665a0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:37.695470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955f9f-1dbd-4ee8-a2e4-a382a665a0bb service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:37.695756 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955f9f-1dbd-4ee8-a2e4-a382a665a0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:37.696035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955f9f-1dbd-4ee8-a2e4-a382a665a0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:37.697871 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-15955f9f-1dbd-4ee8-a2e4-a382a665a0bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:37.768128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15955f9f-1dbd-4ee8-a2e4-a382a665a0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:37.769710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15955f9f-1dbd-4ee8-a2e4-a382a665a0bb service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:37.780370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1042/2067] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:38.575086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076790e-614b-4e98-89e8-a88d34550fc5 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:38.642050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076790e-614b-4e98-89e8-a88d34550fc5 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:38.646678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:38.647283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:38.647565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:38.647878 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] PATH_INFO: `/v3/users/adbbf48a0b014610abb533bd904da689` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:38.649090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adbbf48a0b014610abb533bd904da689)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:38.651473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:38.701007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adbbf48a0b014610abb533bd904da689'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:38.708485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adbbf48a0b014610abb533bd904da689'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:38.709193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adbbf48a0b014610abb533bd904da689'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:38.709821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adbbf48a0b014610abb533bd904da689'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:38.717662 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adbbf48a0b014610abb533bd904da689'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:38.735556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1026/2068] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:38 2022] DELETE /identity/v3/users/adbbf48a0b014610abb533bd904da689 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:38.740267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d602f79-8914-46b0-acfb-eb945a4890e2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:38.740798 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d602f79-8914-46b0-acfb-eb945a4890e2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:38.741006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d602f79-8914-46b0-acfb-eb945a4890e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:38.760195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-2d602f79-8914-46b0-acfb-eb945a4890e2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:38.761518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2d602f79-8914-46b0-acfb-eb945a4890e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:38.779948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:38.822500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1043/2069] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:38.832951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:38.834463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:38.888494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:38.893039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e15fc2-0d6a-4123-869a-0f86fe92c810 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:38.898072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf145c4-7981-4cea-b098-5b911ee3cf8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:38.898739 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf145c4-7981-4cea-b098-5b911ee3cf8b service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:38.899248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf145c4-7981-4cea-b098-5b911ee3cf8b service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:38.899742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf145c4-7981-4cea-b098-5b911ee3cf8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:38.917234 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf145c4-7981-4cea-b098-5b911ee3cf8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:38.920105 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf145c4-7981-4cea-b098-5b911ee3cf8b service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:38.941293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1027/2070] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:47:39.083610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d602f79-8914-46b0-acfb-eb945a4890e2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:39.085662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2d602f79-8914-46b0-acfb-eb945a4890e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:39.100421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d602f79-8914-46b0-acfb-eb945a4890e2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:39.104424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:39.105039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:39.105318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:39.105596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:39.107321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:39.120686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:39.122115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:39.132861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1044/2071] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:39.601595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf145c4-7981-4cea-b098-5b911ee3cf8b service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:39.618387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf145c4-7981-4cea-b098-5b911ee3cf8b service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:39.624306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a366994-f771-4c38-bc8b-aa06587e6d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:39.625113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a366994-f771-4c38-bc8b-aa06587e6d41 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:39.625326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a366994-f771-4c38-bc8b-aa06587e6d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:39.625663 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a366994-f771-4c38-bc8b-aa06587e6d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:39.628301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8a366994-f771-4c38-bc8b-aa06587e6d41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:39.641327 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a366994-f771-4c38-bc8b-aa06587e6d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:39.643242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a366994-f771-4c38-bc8b-aa06587e6d41 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:39.654799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1028/2072] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:39.855254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:39.857548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:39.870590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc42e-e4c2-4e62-aed7-caab7576ec85 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:39.876648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:39.877943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:39.878464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:39.878746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] PATH_INFO: `/v3/projects/72ee5752a7e64363a42c9d1f260b1f3d` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:39.879956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ee5752a7e64363a42c9d1f260b1f3d)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:39.882146 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:39.985730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ee5752a7e64363a42c9d1f260b1f3d', 'request_id': 'req-b5531333-4787-49e7-b888-bbe29d334387'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:39.993137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ee5752a7e64363a42c9d1f260b1f3d', 'request_id': 'req-b5531333-4787-49e7-b888-bbe29d334387'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:40.007638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ee5752a7e64363a42c9d1f260b1f3d', 'request_id': 'req-b5531333-4787-49e7-b888-bbe29d334387'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:40.027504 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] The token cache is being invalidate because project 72ee5752a7e64363a42c9d1f260b1f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:40.027982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:40.028876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:40.036570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1045/2073] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:39 2022] DELETE /identity/v3/projects/72ee5752a7e64363a42c9d1f260b1f3d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:40.037714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a366994-f771-4c38-bc8b-aa06587e6d41 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:40.049015 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a366994-f771-4c38-bc8b-aa06587e6d41 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:40.173991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:40.175028 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:40.175513 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:40.176208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] PATH_INFO: `/v3/users/d42774d511c947a38347f93e72c31735` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:40.178785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d42774d511c947a38347f93e72c31735)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:40.181626 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:40.260421 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd42774d511c947a38347f93e72c31735'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:40.270052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd42774d511c947a38347f93e72c31735'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:40.270747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd42774d511c947a38347f93e72c31735'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:40.271883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd42774d511c947a38347f93e72c31735'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:40.286546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-a6dd8baa-251f-44e2-93a1-07e43f463da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd42774d511c947a38347f93e72c31735'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:40.319032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1029/2074] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:40 2022] DELETE /identity/v3/users/d42774d511c947a38347f93e72c31735 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:40.925391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:40.927141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:40.943217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5531333-4787-49e7-b888-bbe29d334387 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:41.038004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:41.038411 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:41.038986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:41.039544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] PATH_INFO: `/v3/projects/cbbd23918e7648ebbcc13dd1e4f48b0f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:41.041134 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbbd23918e7648ebbcc13dd1e4f48b0f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:41.043853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:41.145934 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:41.165839 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbbd23918e7648ebbcc13dd1e4f48b0f', 'request_id': 'req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:41.174666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbbd23918e7648ebbcc13dd1e4f48b0f', 'request_id': 'req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:41.190529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbbd23918e7648ebbcc13dd1e4f48b0f', 'request_id': 'req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:41.213806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] The token cache is being invalidate because project cbbd23918e7648ebbcc13dd1e4f48b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:41.214174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:41.214804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e50cf653-4921-4a7a-ae2e-98d145a4bbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:41.216857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1046/2075] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:40 2022] DELETE /identity/v3/projects/cbbd23918e7648ebbcc13dd1e4f48b0f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:41.223084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca00bc-0398-4db1-b4a8-6a434cbccd7e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:41.223629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca00bc-0398-4db1-b4a8-6a434cbccd7e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:41.223629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca00bc-0398-4db1-b4a8-6a434cbccd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:41.260628 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-43ca00bc-0398-4db1-b4a8-6a434cbccd7e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:41.261974 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-43ca00bc-0398-4db1-b4a8-6a434cbccd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:41.409299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:41.432666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1030/2076] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:41.441337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de58f7-c829-45a1-ab9b-093550fa120e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:41.441622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de58f7-c829-45a1-ab9b-093550fa120e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:41.441902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de58f7-c829-45a1-ab9b-093550fa120e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:41.486248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-a3de58f7-c829-45a1-ab9b-093550fa120e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:41.487505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a3de58f7-c829-45a1-ab9b-093550fa120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:41.496452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1047/2077] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:41.500394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca00bc-0398-4db1-b4a8-6a434cbccd7e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:41.520794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca00bc-0398-4db1-b4a8-6a434cbccd7e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:41.526156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51908df-f0b1-4fcb-a406-3ce7205dabb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:41.526776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51908df-f0b1-4fcb-a406-3ce7205dabb3 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:41.527057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51908df-f0b1-4fcb-a406-3ce7205dabb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:41.527334 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51908df-f0b1-4fcb-a406-3ce7205dabb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:41.528007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51908df-f0b1-4fcb-a406-3ce7205dabb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:41.528300 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51908df-f0b1-4fcb-a406-3ce7205dabb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:41.537642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51908df-f0b1-4fcb-a406-3ce7205dabb3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:41.560630 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1031/2078] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:47:41.566473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de58f7-c829-45a1-ab9b-093550fa120e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:41.580906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de58f7-c829-45a1-ab9b-093550fa120e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:41.595051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732dc3a-e6c6-4038-9a95-c1217b7fa5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:41.596154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732dc3a-e6c6-4038-9a95-c1217b7fa5fb admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:41.596154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732dc3a-e6c6-4038-9a95-c1217b7fa5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:41.596396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732dc3a-e6c6-4038-9a95-c1217b7fa5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:41.596964 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732dc3a-e6c6-4038-9a95-c1217b7fa5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:41.599390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732dc3a-e6c6-4038-9a95-c1217b7fa5fb admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:41.665624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1048/2079] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:41 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:41.667447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51908df-f0b1-4fcb-a406-3ce7205dabb3 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:41.683287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51908df-f0b1-4fcb-a406-3ce7205dabb3 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:41.814603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d0d8-af42-4b46-902c-ba41d784cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:41.814887 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d0d8-af42-4b46-902c-ba41d784cace admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:41.815187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d0d8-af42-4b46-902c-ba41d784cace admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:41.815447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d0d8-af42-4b46-902c-ba41d784cace admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:41.818566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5d0d8-af42-4b46-902c-ba41d784cace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:41.820047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5d0d8-af42-4b46-902c-ba41d784cace admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:41.849811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1032/2080] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:41 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:41.851063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732dc3a-e6c6-4038-9a95-c1217b7fa5fb admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:41.863659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732dc3a-e6c6-4038-9a95-c1217b7fa5fb admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:41.868946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceac8b-059c-40e7-8e3a-f9c41c083611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:41.869559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceac8b-059c-40e7-8e3a-f9c41c083611 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:41.869843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceac8b-059c-40e7-8e3a-f9c41c083611 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:41.870488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceac8b-059c-40e7-8e3a-f9c41c083611 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:41.870909 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afceac8b-059c-40e7-8e3a-f9c41c083611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:41.873456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afceac8b-059c-40e7-8e3a-f9c41c083611 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:41.891085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1049/2081] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:41.893235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d0d8-af42-4b46-902c-ba41d784cace admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:41.909128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d0d8-af42-4b46-902c-ba41d784cace admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:41.918770 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6af30b-805d-454e-b5d1-7194d8918cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:41.919607 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6af30b-805d-454e-b5d1-7194d8918cfb admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:41.919897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6af30b-805d-454e-b5d1-7194d8918cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:41.920171 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6af30b-805d-454e-b5d1-7194d8918cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:41.920908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6af30b-805d-454e-b5d1-7194d8918cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:41.923829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6af30b-805d-454e-b5d1-7194d8918cfb admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:41.940982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1033/2082] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:41.942242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceac8b-059c-40e7-8e3a-f9c41c083611 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:41.957151 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceac8b-059c-40e7-8e3a-f9c41c083611 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:41.962856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82087db7-fdab-4470-b825-dd52b825c67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:41.963775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82087db7-fdab-4470-b825-dd52b825c67e admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:41.964431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82087db7-fdab-4470-b825-dd52b825c67e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:41.964431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82087db7-fdab-4470-b825-dd52b825c67e admin admin] PATH_INFO: `/v3/projects/bb85f08631ce432984f8c990b00eb443/users/66756b19816d4ac5867b2ea1f7fbe1e3/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:41.987023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82087db7-fdab-4470-b825-dd52b825c67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb85f08631ce432984f8c990b00eb443, user_id=66756b19816d4ac5867b2ea1f7fbe1e3, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:41.990486 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82087db7-fdab-4470-b825-dd52b825c67e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:42.020944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1050/2083] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:41 2022] PUT /identity/v3/projects/bb85f08631ce432984f8c990b00eb443/users/66756b19816d4ac5867b2ea1f7fbe1e3/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:42.099244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e2777-7d42-4319-82d2-9a3c75852090 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:42.099555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e2777-7d42-4319-82d2-9a3c75852090 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:42.099959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e2777-7d42-4319-82d2-9a3c75852090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:42.166198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-fa2e2777-7d42-4319-82d2-9a3c75852090 None None] MFA Rules not processed for user `66756b19816d4ac5867b2ea1f7fbe1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:42.167869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fa2e2777-7d42-4319-82d2-9a3c75852090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:42.318196 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1034/2084] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:42.326022 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82087db7-fdab-4470-b825-dd52b825c67e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:42.328107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-82087db7-fdab-4470-b825-dd52b825c67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:42.415638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82087db7-fdab-4470-b825-dd52b825c67e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:42.420091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900b86f-ae95-4f84-951f-f11af12d6e89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:42.420760 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900b86f-ae95-4f84-951f-f11af12d6e89 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:42.421045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900b86f-ae95-4f84-951f-f11af12d6e89 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:42.421324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900b86f-ae95-4f84-951f-f11af12d6e89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:42.435550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900b86f-ae95-4f84-951f-f11af12d6e89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:42.437527 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900b86f-ae95-4f84-951f-f11af12d6e89 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:42.452661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1051/2085] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:45.391496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e2777-7d42-4319-82d2-9a3c75852090 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:45.393900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fa2e2777-7d42-4319-82d2-9a3c75852090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:45.491532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:45.513165 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e2777-7d42-4319-82d2-9a3c75852090 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:45.520236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:45.521386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:45.521884 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:45.522404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] PATH_INFO: `/v3/users/4d74d7fd62cd464183f9df4ca581833c` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:45.523889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d74d7fd62cd464183f9df4ca581833c)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:45.526333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:45.611131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d74d7fd62cd464183f9df4ca581833c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:45.621860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d74d7fd62cd464183f9df4ca581833c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:45.622641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d74d7fd62cd464183f9df4ca581833c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:45.623332 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d74d7fd62cd464183f9df4ca581833c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:45.637149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d74d7fd62cd464183f9df4ca581833c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:45.688226 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1035/2086] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:45 2022] DELETE /identity/v3/users/4d74d7fd62cd464183f9df4ca581833c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:46.382885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900b86f-ae95-4f84-951f-f11af12d6e89 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:46.398658 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900b86f-ae95-4f84-951f-f11af12d6e89 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:46.415536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:46.417429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:46.486903 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:46.487805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:46.488269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:46.488744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] PATH_INFO: `/v3/users/66756b19816d4ac5867b2ea1f7fbe1e3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:46.490270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66756b19816d4ac5867b2ea1f7fbe1e3)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:46.492630 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:46.499434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:46.538704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49a1c0-3f2e-41cd-a3ca-9a259d6fba14 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:46.546364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:46.548545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:46.549579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:46.550055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] PATH_INFO: `/v3/projects/a3d7fadfead1456792446f71c465bc0c` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:46.559784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d7fadfead1456792446f71c465bc0c)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:46.563235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:46.605715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66756b19816d4ac5867b2ea1f7fbe1e3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.618092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66756b19816d4ac5867b2ea1f7fbe1e3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.619035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66756b19816d4ac5867b2ea1f7fbe1e3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.620152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66756b19816d4ac5867b2ea1f7fbe1e3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.630410 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-07a61b57-ddfe-414a-9a6e-86f80a829ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66756b19816d4ac5867b2ea1f7fbe1e3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.658345 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d7fadfead1456792446f71c465bc0c', 'request_id': 'req-ea70ed68-934a-44b9-acdd-e0d630f7fa85'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.664795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d7fadfead1456792446f71c465bc0c', 'request_id': 'req-ea70ed68-934a-44b9-acdd-e0d630f7fa85'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.668056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1052/2087] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:46 2022] DELETE /identity/v3/users/66756b19816d4ac5867b2ea1f7fbe1e3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:46.676886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6886b-82b0-4ac8-b655-d56b1426ac95 None None] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:46.681875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d7fadfead1456792446f71c465bc0c', 'request_id': 'req-ea70ed68-934a-44b9-acdd-e0d630f7fa85'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.682535 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6886b-82b0-4ac8-b655-d56b1426ac95 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:46.682899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6886b-82b0-4ac8-b655-d56b1426ac95 None None] PATH_INFO: `/` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:46.684522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1053/2088] 10.222.0.13 () {54 vars in 921 bytes} [Thu Oct 20 14:47:46 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 14:47:46.686193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f07da-a499-49e7-af19-6fb07fa3a3e5 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:46.686546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f07da-a499-49e7-af19-6fb07fa3a3e5 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:46.686910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f07da-a499-49e7-af19-6fb07fa3a3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:46.711177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] The token cache is being invalidate because project a3d7fadfead1456792446f71c465bc0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:46.711473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.712128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ea70ed68-934a-44b9-acdd-e0d630f7fa85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.714030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1036/2089] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:46 2022] DELETE /identity/v3/projects/a3d7fadfead1456792446f71c465bc0c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:46.719076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ba7b7-bbe3-40b6-a6f8-912746f2c8cb None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:46.719463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ba7b7-bbe3-40b6-a6f8-912746f2c8cb None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:46.719739 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ba7b7-bbe3-40b6-a6f8-912746f2c8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:46.728428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:46.729998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ef4f07da-a499-49e7-af19-6fb07fa3a3e5 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:46.731290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ef4f07da-a499-49e7-af19-6fb07fa3a3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:46.747424 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-1f4ba7b7-bbe3-40b6-a6f8-912746f2c8cb None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:46.749654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1f4ba7b7-bbe3-40b6-a6f8-912746f2c8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:46.825180 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1054/2090] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:46.826419 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7cc14-ce27-40e5-adb7-907b21f0ca01 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:46.826708 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7cc14-ce27-40e5-adb7-907b21f0ca01 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:46.827012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7cc14-ce27-40e5-adb7-907b21f0ca01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:46.845742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1037/2091] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:46.846578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ba7b7-bbe3-40b6-a6f8-912746f2c8cb None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:46.847852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1f4ba7b7-bbe3-40b6-a6f8-912746f2c8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:46.858681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1ed7cc14-ce27-40e5-adb7-907b21f0ca01 None None] MFA Rules not processed for user `fc18a78229064e8d8ea4321c3c23c4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:46.860025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1ed7cc14-ce27-40e5-adb7-907b21f0ca01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:46.860184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ba7b7-bbe3-40b6-a6f8-912746f2c8cb None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:46.864872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:46.865492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:46.865767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:46.866038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] PATH_INFO: `/v3/users/e0d4be0e0f744242a323ac9694aef9ad` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:46.867167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d4be0e0f744242a323ac9694aef9ad)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:46.869112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:46.917541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d4be0e0f744242a323ac9694aef9ad'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.925049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d4be0e0f744242a323ac9694aef9ad'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.925860 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d4be0e0f744242a323ac9694aef9ad'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.926648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d4be0e0f744242a323ac9694aef9ad'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.930473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1055/2092] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Oct 20 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:46.931594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7cc14-ce27-40e5-adb7-907b21f0ca01 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:46.933009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1ed7cc14-ce27-40e5-adb7-907b21f0ca01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:46.941970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e11e5fe6-ec9a-487d-a411-6efaadf38210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d4be0e0f744242a323ac9694aef9ad'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:46.957360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:46.963918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1038/2093] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:46 2022] DELETE /identity/v3/users/e0d4be0e0f744242a323ac9694aef9ad => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:46.966042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddfa5c-8de8-45ec-b0f0-dd4da054b074 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:46.966344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddfa5c-8de8-45ec-b0f0-dd4da054b074 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:46.966651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddfa5c-8de8-45ec-b0f0-dd4da054b074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:46.993205 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-73ddfa5c-8de8-45ec-b0f0-dd4da054b074 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:46.994958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-73ddfa5c-8de8-45ec-b0f0-dd4da054b074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:46.999752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7cc14-ce27-40e5-adb7-907b21f0ca01 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.065008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1039/2094] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:47.065776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddfa5c-8de8-45ec-b0f0-dd4da054b074 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.069592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-73ddfa5c-8de8-45ec-b0f0-dd4da054b074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.072845 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a0785-da2c-4ce0-b8c8-5594e020a9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.073672 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a0785-da2c-4ce0-b8c8-5594e020a9bb service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.073970 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a0785-da2c-4ce0-b8c8-5594e020a9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.074250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a0785-da2c-4ce0-b8c8-5594e020a9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.076121 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-731a0785-da2c-4ce0-b8c8-5594e020a9bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.089975 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a0785-da2c-4ce0-b8c8-5594e020a9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.091425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a0785-da2c-4ce0-b8c8-5594e020a9bb service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.092454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:47.102090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1056/2095] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:47.102976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a0785-da2c-4ce0-b8c8-5594e020a9bb service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.114465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a0785-da2c-4ce0-b8c8-5594e020a9bb service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.122755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04df60-6c65-4218-aaab-787d797ebe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.124126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04df60-6c65-4218-aaab-787d797ebe2e admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.124600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04df60-6c65-4218-aaab-787d797ebe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.125165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04df60-6c65-4218-aaab-787d797ebe2e admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.127648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04df60-6c65-4218-aaab-787d797ebe2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:47.128133 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04df60-6c65-4218-aaab-787d797ebe2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.130667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04df60-6c65-4218-aaab-787d797ebe2e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.150157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1057/2096] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:47:47.151595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04df60-6c65-4218-aaab-787d797ebe2e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.163787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04df60-6c65-4218-aaab-787d797ebe2e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.169666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93031ab-5d3f-4061-a568-5af972021bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.170340 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93031ab-5d3f-4061-a568-5af972021bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.170647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93031ab-5d3f-4061-a568-5af972021bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.170947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93031ab-5d3f-4061-a568-5af972021bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.171625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93031ab-5d3f-4061-a568-5af972021bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.173939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93031ab-5d3f-4061-a568-5af972021bb3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.185098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddfa5c-8de8-45ec-b0f0-dd4da054b074 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.192374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc18a78229064e8d8ea4321c3c23c4ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.192792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.193079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.193376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.195385 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.208487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.210518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.223180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1040/2097] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:47.223861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.225523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.238474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1058/2098] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:47.239335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93031ab-5d3f-4061-a568-5af972021bb3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.243606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5567cf-61c7-48e8-bebc-399bab3afade service placement] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.321724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.322526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.322651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.322989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] PATH_INFO: `/v3/projects/8d8d1dfd3c714794a3fca61b2c6b093e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.324211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d8d1dfd3c714794a3fca61b2c6b093e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.326262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.331308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93031ab-5d3f-4061-a568-5af972021bb3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.341849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b89e60-3e33-4de9-aa5a-e0dea69d117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.341849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b89e60-3e33-4de9-aa5a-e0dea69d117b admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.341849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b89e60-3e33-4de9-aa5a-e0dea69d117b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.341849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b89e60-3e33-4de9-aa5a-e0dea69d117b admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.342297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b89e60-3e33-4de9-aa5a-e0dea69d117b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.342297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b89e60-3e33-4de9-aa5a-e0dea69d117b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.371956 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1059/2099] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:47.376626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b89e60-3e33-4de9-aa5a-e0dea69d117b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.386719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b89e60-3e33-4de9-aa5a-e0dea69d117b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.398465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d8d1dfd3c714794a3fca61b2c6b093e', 'request_id': 'req-2d766455-9631-4d2e-a56f-037b9b5e8bca'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.406413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d8d1dfd3c714794a3fca61b2c6b093e', 'request_id': 'req-2d766455-9631-4d2e-a56f-037b9b5e8bca'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.416241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d8d1dfd3c714794a3fca61b2c6b093e', 'request_id': 'req-2d766455-9631-4d2e-a56f-037b9b5e8bca'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.421314 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:47.432596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] The token cache is being invalidate because project 8d8d1dfd3c714794a3fca61b2c6b093e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:47.433027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.433727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-2d766455-9631-4d2e-a56f-037b9b5e8bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.435608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1041/2100] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:47 2022] DELETE /identity/v3/projects/8d8d1dfd3c714794a3fca61b2c6b093e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:47.440718 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da734531-ea0a-4751-afb9-e9438a29619c None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.441012 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da734531-ea0a-4751-afb9-e9438a29619c None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.441295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da734531-ea0a-4751-afb9-e9438a29619c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.459847 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472467c3-fd8b-467c-898b-d2b7857f5b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.460506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472467c3-fd8b-467c-898b-d2b7857f5b4c admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.460780 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472467c3-fd8b-467c-898b-d2b7857f5b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.461045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472467c3-fd8b-467c-898b-d2b7857f5b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.461694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472467c3-fd8b-467c-898b-d2b7857f5b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.463319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472467c3-fd8b-467c-898b-d2b7857f5b4c admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.472807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-da734531-ea0a-4751-afb9-e9438a29619c None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:47.474041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-da734531-ea0a-4751-afb9-e9438a29619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.474662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1060/2101] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:47.477294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472467c3-fd8b-467c-898b-d2b7857f5b4c admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.479360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-472467c3-fd8b-467c-898b-d2b7857f5b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.489634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1042/2102] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:47.489634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da734531-ea0a-4751-afb9-e9438a29619c None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.489634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-da734531-ea0a-4751-afb9-e9438a29619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.493859 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472467c3-fd8b-467c-898b-d2b7857f5b4c admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.496994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da734531-ea0a-4751-afb9-e9438a29619c None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.498938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.499555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.499832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.500107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] PATH_INFO: `/v3/projects/bb85f08631ce432984f8c990b00eb443` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.502150 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb85f08631ce432984f8c990b00eb443)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.502382 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8eca8c-a5ea-45b5-905a-2821c124c7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.502972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8eca8c-a5ea-45b5-905a-2821c124c7bd admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.503248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8eca8c-a5ea-45b5-905a-2821c124c7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.503518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8eca8c-a5ea-45b5-905a-2821c124c7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.504049 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.504155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8eca8c-a5ea-45b5-905a-2821c124c7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.505999 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8eca8c-a5ea-45b5-905a-2821c124c7bd admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.517827 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1043/2103] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:47.518992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0101520-b581-4715-baef-8fc2484f2171 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.519857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0101520-b581-4715-baef-8fc2484f2171 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.520134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0101520-b581-4715-baef-8fc2484f2171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.540959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b0101520-b581-4715-baef-8fc2484f2171 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:47.542169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b0101520-b581-4715-baef-8fc2484f2171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.555134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:47.555866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb85f08631ce432984f8c990b00eb443', 'request_id': 'req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.561883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb85f08631ce432984f8c990b00eb443', 'request_id': 'req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.570089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb85f08631ce432984f8c990b00eb443', 'request_id': 'req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.583936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] The token cache is being invalidate because project bb85f08631ce432984f8c990b00eb443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:47.584610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.585282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:47.587196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1061/2104] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:47 2022] DELETE /identity/v3/projects/bb85f08631ce432984f8c990b00eb443 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:47.588428 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.589883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.605613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1044/2105] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:47.608809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9802-ac04-4828-baf8-fb168a746a79 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.608809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9802-ac04-4828-baf8-fb168a746a79 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.608809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9802-ac04-4828-baf8-fb168a746a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.627065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-083d9802-ac04-4828-baf8-fb168a746a79 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:47.628272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-083d9802-ac04-4828-baf8-fb168a746a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.636248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1045/2106] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:47.636894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d9802-ac04-4828-baf8-fb168a746a79 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.646306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d9802-ac04-4828-baf8-fb168a746a79 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.651401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955aa7c1-e085-47f3-a271-cbe1e5277134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.652024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955aa7c1-e085-47f3-a271-cbe1e5277134 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.652306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955aa7c1-e085-47f3-a271-cbe1e5277134 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.652615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955aa7c1-e085-47f3-a271-cbe1e5277134 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.653377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955aa7c1-e085-47f3-a271-cbe1e5277134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:47.653592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955aa7c1-e085-47f3-a271-cbe1e5277134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.655263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955aa7c1-e085-47f3-a271-cbe1e5277134 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.659170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9b179-ce46-4243-a5b2-4ae86c5b4d04 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.664020 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9662f-04e4-42c4-8ed9-67d1b29631d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.664705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9662f-04e4-42c4-8ed9-67d1b29631d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.664985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9662f-04e4-42c4-8ed9-67d1b29631d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.665254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9662f-04e4-42c4-8ed9-67d1b29631d2 admin admin] PATH_INFO: `/v3/projects/9b4493ebf93b4305bf4e90c5d36013c5/users/717d38be0fed4071953fdf4a1aa9f198/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.667739 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1046/2107] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:47:47.669279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41246803-a8dd-41f5-b0ea-0a857d6c45fd None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.669570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41246803-a8dd-41f5-b0ea-0a857d6c45fd None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.669852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41246803-a8dd-41f5-b0ea-0a857d6c45fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.679784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9662f-04e4-42c4-8ed9-67d1b29631d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4493ebf93b4305bf4e90c5d36013c5, user_id=717d38be0fed4071953fdf4a1aa9f198, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.682450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9662f-04e4-42c4-8ed9-67d1b29631d2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.689950 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-41246803-a8dd-41f5-b0ea-0a857d6c45fd None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:47.691155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-41246803-a8dd-41f5-b0ea-0a857d6c45fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.697009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1062/2108] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:47 2022] PUT /identity/v3/projects/9b4493ebf93b4305bf4e90c5d36013c5/users/717d38be0fed4071953fdf4a1aa9f198/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:47.697777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9662f-04e4-42c4-8ed9-67d1b29631d2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.737321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:47.758584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1047/2109] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:47.759833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033789d-1e95-47e4-80cb-edcbbc4fe481 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.760125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033789d-1e95-47e4-80cb-edcbbc4fe481 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.760439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033789d-1e95-47e4-80cb-edcbbc4fe481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.765735 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9662f-04e4-42c4-8ed9-67d1b29631d2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.771054 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e92115-ade9-4426-ab18-90a073e5924f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.771650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e92115-ade9-4426-ab18-90a073e5924f admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.771925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e92115-ade9-4426-ab18-90a073e5924f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.772194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e92115-ade9-4426-ab18-90a073e5924f admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.772846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e92115-ade9-4426-ab18-90a073e5924f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.774732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e92115-ade9-4426-ab18-90a073e5924f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.787676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-7033789d-1e95-47e4-80cb-edcbbc4fe481 None None] MFA Rules not processed for user `717d38be0fed4071953fdf4a1aa9f198`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:47.788903 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7033789d-1e95-47e4-80cb-edcbbc4fe481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:47.800833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1063/2110] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:47.801892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e92115-ade9-4426-ab18-90a073e5924f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.858983 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1048/2111] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:47.859928 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033789d-1e95-47e4-80cb-edcbbc4fe481 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.870901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033789d-1e95-47e4-80cb-edcbbc4fe481 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.872676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e92115-ade9-4426-ab18-90a073e5924f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.876308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8649000-ac9c-4192-9f46-9125cafe0e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.876942 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8649000-ac9c-4192-9f46-9125cafe0e6a admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.877215 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8649000-ac9c-4192-9f46-9125cafe0e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.877486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8649000-ac9c-4192-9f46-9125cafe0e6a admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.877918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0686e-c895-465c-991b-2eef15676e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.878115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8649000-ac9c-4192-9f46-9125cafe0e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.878550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0686e-c895-465c-991b-2eef15676e61 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.878793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0686e-c895-465c-991b-2eef15676e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.879070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0686e-c895-465c-991b-2eef15676e61 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.879842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0686e-c895-465c-991b-2eef15676e61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:47.880053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8649000-ac9c-4192-9f46-9125cafe0e6a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.880175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0686e-c895-465c-991b-2eef15676e61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.881681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0686e-c895-465c-991b-2eef15676e61 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.896899 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1064/2112] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:47:47.903040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0686e-c895-465c-991b-2eef15676e61 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.911224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1049/2113] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/users => generated 286 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:47.911224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8649000-ac9c-4192-9f46-9125cafe0e6a admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.915392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0686e-c895-465c-991b-2eef15676e61 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.920247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff161fb4-a968-4ec8-afe5-e657906ff4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.921007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161fb4-a968-4ec8-afe5-e657906ff4d8 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.921371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161fb4-a968-4ec8-afe5-e657906ff4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.921474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8649000-ac9c-4192-9f46-9125cafe0e6a admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:47.921822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161fb4-a968-4ec8-afe5-e657906ff4d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.922563 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff161fb4-a968-4ec8-afe5-e657906ff4d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.924886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff161fb4-a968-4ec8-afe5-e657906ff4d8 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.926863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85f814-b1d5-4e07-8d7f-0b8e3e7b6954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:47.927543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85f814-b1d5-4e07-8d7f-0b8e3e7b6954 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:47.927898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85f814-b1d5-4e07-8d7f-0b8e3e7b6954 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:47.928431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85f814-b1d5-4e07-8d7f-0b8e3e7b6954 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:47.929953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85f814-b1d5-4e07-8d7f-0b8e3e7b6954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:47.931076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85f814-b1d5-4e07-8d7f-0b8e3e7b6954 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:47.943657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:47.967348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1065/2114] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:47.967348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff161fb4-a968-4ec8-afe5-e657906ff4d8 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:47.975039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1050/2115] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:47.975039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85f814-b1d5-4e07-8d7f-0b8e3e7b6954 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.040969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff161fb4-a968-4ec8-afe5-e657906ff4d8 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.053008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85f814-b1d5-4e07-8d7f-0b8e3e7b6954 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.058930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29d07-7c96-47bb-a6f9-e3d2072e17f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.059545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29d07-7c96-47bb-a6f9-e3d2072e17f2 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.059821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29d07-7c96-47bb-a6f9-e3d2072e17f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.060093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29d07-7c96-47bb-a6f9-e3d2072e17f2 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.060809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29d07-7c96-47bb-a6f9-e3d2072e17f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.064184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29d07-7c96-47bb-a6f9-e3d2072e17f2 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.086113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:48.087149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1051/2116] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:47 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:48.090439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29d07-7c96-47bb-a6f9-e3d2072e17f2 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.100479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29d07-7c96-47bb-a6f9-e3d2072e17f2 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.105292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b1d27-1f1f-4860-868f-ded396927505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.105876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b1d27-1f1f-4860-868f-ded396927505 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.106153 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b1d27-1f1f-4860-868f-ded396927505 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.106426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b1d27-1f1f-4860-868f-ded396927505 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.107024 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f044fb1-f4f6-4384-a816-df80d43cfd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.107152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b1d27-1f1f-4860-868f-ded396927505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.107636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f044fb1-f4f6-4384-a816-df80d43cfd2d admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.107916 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f044fb1-f4f6-4384-a816-df80d43cfd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.108251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f044fb1-f4f6-4384-a816-df80d43cfd2d admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.108983 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f044fb1-f4f6-4384-a816-df80d43cfd2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.109100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b1d27-1f1f-4860-868f-ded396927505 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.111157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f044fb1-f4f6-4384-a816-df80d43cfd2d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.122967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1066/2117] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:48.124023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f044fb1-f4f6-4384-a816-df80d43cfd2d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.134852 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f044fb1-f4f6-4384-a816-df80d43cfd2d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.135459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1052/2118] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:48 2022] POST /identity/v3/users => generated 285 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:48.136425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b1d27-1f1f-4860-868f-ded396927505 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.139963 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a8763-937d-4c80-9293-779bbfab21b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.140598 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a8763-937d-4c80-9293-779bbfab21b7 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.140882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a8763-937d-4c80-9293-779bbfab21b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.141221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a8763-937d-4c80-9293-779bbfab21b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.142052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a8763-937d-4c80-9293-779bbfab21b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.143665 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a8763-937d-4c80-9293-779bbfab21b7 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.146562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b1d27-1f1f-4860-868f-ded396927505 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.151469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4120823-10ce-4bb3-867d-52142da11fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.152090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4120823-10ce-4bb3-867d-52142da11fed admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.152386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4120823-10ce-4bb3-867d-52142da11fed admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.152662 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4120823-10ce-4bb3-867d-52142da11fed admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.153344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4120823-10ce-4bb3-867d-52142da11fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.154953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1067/2119] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:48.155710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4120823-10ce-4bb3-867d-52142da11fed admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.155830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a8763-937d-4c80-9293-779bbfab21b7 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.168238 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1053/2120] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:48.168431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a8763-937d-4c80-9293-779bbfab21b7 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.168714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4120823-10ce-4bb3-867d-52142da11fed admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.176104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2d11-bcd0-4e7e-98d9-7156c223b5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.176736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2d11-bcd0-4e7e-98d9-7156c223b5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.177018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2d11-bcd0-4e7e-98d9-7156c223b5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.177300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2d11-bcd0-4e7e-98d9-7156c223b5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.177944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2d11-bcd0-4e7e-98d9-7156c223b5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.179603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2d11-bcd0-4e7e-98d9-7156c223b5e3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.181177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4120823-10ce-4bb3-867d-52142da11fed admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.186122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254120e0-0e51-483b-accb-f867050d995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.187008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254120e0-0e51-483b-accb-f867050d995c admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.187497 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254120e0-0e51-483b-accb-f867050d995c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.187906 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254120e0-0e51-483b-accb-f867050d995c admin admin] PATH_INFO: `/v3/projects/ccce311da4f64971b88759cc7e5d9baa/users/40833e9a11de42a5a0b4c6f460763f9a/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.192423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1068/2121] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:48.193222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2d11-bcd0-4e7e-98d9-7156c223b5e3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.203278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2d11-bcd0-4e7e-98d9-7156c223b5e3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.203946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254120e0-0e51-483b-accb-f867050d995c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccce311da4f64971b88759cc7e5d9baa, user_id=40833e9a11de42a5a0b4c6f460763f9a, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.207004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254120e0-0e51-483b-accb-f867050d995c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.208415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ccaf5-2734-4231-93ec-01c5c2c3a508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.209053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ccaf5-2734-4231-93ec-01c5c2c3a508 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.209332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ccaf5-2734-4231-93ec-01c5c2c3a508 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.209603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ccaf5-2734-4231-93ec-01c5c2c3a508 admin admin] PATH_INFO: `/v3/projects/36dbbca5eec64d4f83d4d1fc8dc7bdf3/users/3387d3e9798f453ba6633b6c822b31db/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.220792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1054/2122] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:48 2022] PUT /identity/v3/projects/ccce311da4f64971b88759cc7e5d9baa/users/40833e9a11de42a5a0b4c6f460763f9a/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:48.221612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254120e0-0e51-483b-accb-f867050d995c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.225520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ccaf5-2734-4231-93ec-01c5c2c3a508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36dbbca5eec64d4f83d4d1fc8dc7bdf3, user_id=3387d3e9798f453ba6633b6c822b31db, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.228296 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ccaf5-2734-4231-93ec-01c5c2c3a508 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.243173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1069/2123] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:48 2022] PUT /identity/v3/projects/36dbbca5eec64d4f83d4d1fc8dc7bdf3/users/3387d3e9798f453ba6633b6c822b31db/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:48.243984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ccaf5-2734-4231-93ec-01c5c2c3a508 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.287050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254120e0-0e51-483b-accb-f867050d995c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.310807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ccaf5-2734-4231-93ec-01c5c2c3a508 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.316702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c63450-37b0-414b-872a-35fd1b946cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.317311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63450-37b0-414b-872a-35fd1b946cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.317595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63450-37b0-414b-872a-35fd1b946cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.317878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63450-37b0-414b-872a-35fd1b946cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.318500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:48.318647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c63450-37b0-414b-872a-35fd1b946cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.320179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c63450-37b0-414b-872a-35fd1b946cb3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.333893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1070/2124] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:48.333893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda2929-0146-4783-92e1-8dfff2a56801 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.333893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda2929-0146-4783-92e1-8dfff2a56801 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.333893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda2929-0146-4783-92e1-8dfff2a56801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.361063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d0fd-14ca-477b-adf9-1beaaf005967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.362029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d0fd-14ca-477b-adf9-1beaaf005967 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.362147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-2fda2929-0146-4783-92e1-8dfff2a56801 None None] MFA Rules not processed for user `3387d3e9798f453ba6633b6c822b31db`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:48.362313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d0fd-14ca-477b-adf9-1beaaf005967 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.362586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d0fd-14ca-477b-adf9-1beaaf005967 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.363258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d0fd-14ca-477b-adf9-1beaaf005967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.363374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2fda2929-0146-4783-92e1-8dfff2a56801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:48.364959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d0fd-14ca-477b-adf9-1beaaf005967 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.374877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1055/2125] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:48.375756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d0fd-14ca-477b-adf9-1beaaf005967 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.385310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d0fd-14ca-477b-adf9-1beaaf005967 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.389884 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489e910-5fff-420c-b8fe-72c83957a746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.390509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489e910-5fff-420c-b8fe-72c83957a746 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.391107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489e910-5fff-420c-b8fe-72c83957a746 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.391107 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489e910-5fff-420c-b8fe-72c83957a746 admin admin] PATH_INFO: `/v3/projects/ccce311da4f64971b88759cc7e5d9baa/users/40833e9a11de42a5a0b4c6f460763f9a/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.393148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489e910-5fff-420c-b8fe-72c83957a746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccce311da4f64971b88759cc7e5d9baa, user_id=40833e9a11de42a5a0b4c6f460763f9a, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.395746 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489e910-5fff-420c-b8fe-72c83957a746 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.405814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1056/2126] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:48 2022] PUT /identity/v3/projects/ccce311da4f64971b88759cc7e5d9baa/users/40833e9a11de42a5a0b4c6f460763f9a/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:48.406371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489e910-5fff-420c-b8fe-72c83957a746 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.430202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1071/2127] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:48.431444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d1108-28f8-4c0b-bc49-57628c1edee8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.431724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d1108-28f8-4c0b-bc49-57628c1edee8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.431994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d1108-28f8-4c0b-bc49-57628c1edee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.460437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4e7d1108-28f8-4c0b-bc49-57628c1edee8 None None] MFA Rules not processed for user `40833e9a11de42a5a0b4c6f460763f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:48.461635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4e7d1108-28f8-4c0b-bc49-57628c1edee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:48.465540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:48.480421 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489e910-5fff-420c-b8fe-72c83957a746 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.485366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a288466-19f0-4957-88c7-726602a1613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.486036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a288466-19f0-4957-88c7-726602a1613e admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.486318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a288466-19f0-4957-88c7-726602a1613e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.486599 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a288466-19f0-4957-88c7-726602a1613e admin admin] PATH_INFO: `/v3/projects/ef7734b972424fbc9c292640687f88e4/users/f3194340fe774d72bba97563b73317ec/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.501659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a288466-19f0-4957-88c7-726602a1613e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7734b972424fbc9c292640687f88e4, user_id=f3194340fe774d72bba97563b73317ec, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.504374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a288466-19f0-4957-88c7-726602a1613e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.520198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1057/2128] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:48 2022] PUT /identity/v3/projects/ef7734b972424fbc9c292640687f88e4/users/f3194340fe774d72bba97563b73317ec/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:48.521203 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a288466-19f0-4957-88c7-726602a1613e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.523366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3a288466-19f0-4957-88c7-726602a1613e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:48.537097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1072/2129] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:48.538575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1e7eb-1b3c-4cf6-b863-9fd69fbaa7a7 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.538867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1e7eb-1b3c-4cf6-b863-9fd69fbaa7a7 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.539138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1e7eb-1b3c-4cf6-b863-9fd69fbaa7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.568261 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f2d1e7eb-1b3c-4cf6-b863-9fd69fbaa7a7 None None] MFA Rules not processed for user `f3194340fe774d72bba97563b73317ec`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:48.569572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f2d1e7eb-1b3c-4cf6-b863-9fd69fbaa7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:48.597200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a288466-19f0-4957-88c7-726602a1613e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.603134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c73714-e7eb-4017-b2e1-b5b2557258c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.603979 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c73714-e7eb-4017-b2e1-b5b2557258c0 service nova] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.604265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c73714-e7eb-4017-b2e1-b5b2557258c0 service nova] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.604567 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c73714-e7eb-4017-b2e1-b5b2557258c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.633312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1073/2130] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:48.634066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1e7eb-1b3c-4cf6-b863-9fd69fbaa7a7 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.635519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f2d1e7eb-1b3c-4cf6-b863-9fd69fbaa7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:48.640146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:48.666259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c73714-e7eb-4017-b2e1-b5b2557258c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.667890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c73714-e7eb-4017-b2e1-b5b2557258c0 service nova] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.677457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1058/2131] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:48.678253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c73714-e7eb-4017-b2e1-b5b2557258c0 service nova] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:48.679591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-61c73714-e7eb-4017-b2e1-b5b2557258c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:48.696257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1e7eb-1b3c-4cf6-b863-9fd69fbaa7a7 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.700207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747616a-cc14-4068-9b69-6f1328d980e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.700851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747616a-cc14-4068-9b69-6f1328d980e4 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.701132 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747616a-cc14-4068-9b69-6f1328d980e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.701544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747616a-cc14-4068-9b69-6f1328d980e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.737880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c73714-e7eb-4017-b2e1-b5b2557258c0 service nova] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:48.741640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2c588-cd42-44f1-9d88-43922bdd90e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cacfce5c9d07471cbe1713a4d6aa29f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:48.742248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2c588-cd42-44f1-9d88-43922bdd90e8 service glance] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:48.742527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2c588-cd42-44f1-9d88-43922bdd90e8 service glance] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:48.742800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2c588-cd42-44f1-9d88-43922bdd90e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:48.752765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2c588-cd42-44f1-9d88-43922bdd90e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.754475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2c588-cd42-44f1-9d88-43922bdd90e8 service glance] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.756715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:48.765276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1059/2132] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:48.775236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747616a-cc14-4068-9b69-6f1328d980e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:48.776918 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747616a-cc14-4068-9b69-6f1328d980e4 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:48.788645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1074/2133] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:49.031932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2c588-cd42-44f1-9d88-43922bdd90e8 service glance] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:49.033591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-abb2c588-cd42-44f1-9d88-43922bdd90e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:49.049700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2c588-cd42-44f1-9d88-43922bdd90e8 service glance] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:49.054099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9c932-7188-4ef7-aec2-b1990d50f21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:49.054768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9c932-7188-4ef7-aec2-b1990d50f21d service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:49.055050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9c932-7188-4ef7-aec2-b1990d50f21d service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:49.055422 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9c932-7188-4ef7-aec2-b1990d50f21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:49.118969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9c932-7188-4ef7-aec2-b1990d50f21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:49.120486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9c932-7188-4ef7-aec2-b1990d50f21d service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:49.130900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1060/2134] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:49.362074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747616a-cc14-4068-9b69-6f1328d980e4 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:49.363500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2747616a-cc14-4068-9b69-6f1328d980e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:49.427511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747616a-cc14-4068-9b69-6f1328d980e4 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:49.431079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da685c-51eb-4679-8530-32028a2c0e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:49.431689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da685c-51eb-4679-8530-32028a2c0e83 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:49.431938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da685c-51eb-4679-8530-32028a2c0e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:49.432310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da685c-51eb-4679-8530-32028a2c0e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:49.434008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-57da685c-51eb-4679-8530-32028a2c0e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:49.497914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da685c-51eb-4679-8530-32028a2c0e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:49.499331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da685c-51eb-4679-8530-32028a2c0e83 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:49.509292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1075/2135] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:51.839445 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9c932-7188-4ef7-aec2-b1990d50f21d service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:51.852982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9c932-7188-4ef7-aec2-b1990d50f21d service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:51.857989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:51.858525 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:51.858799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:51.859067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] PATH_INFO: `/v3/users/3387d3e9798f453ba6633b6c822b31db` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:51.860901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3387d3e9798f453ba6633b6c822b31db)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:51.862805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:51.923518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3387d3e9798f453ba6633b6c822b31db'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:51.931903 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3387d3e9798f453ba6633b6c822b31db'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:51.932700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3387d3e9798f453ba6633b6c822b31db'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:51.933305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3387d3e9798f453ba6633b6c822b31db'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:51.941745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3387d3e9798f453ba6633b6c822b31db'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:51.949324 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:51.964612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1061/2136] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:51 2022] DELETE /identity/v3/users/3387d3e9798f453ba6633b6c822b31db => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:51.969029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45332d00-0bec-4379-8673-81821ba2f220 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:51.969655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45332d00-0bec-4379-8673-81821ba2f220 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:51.969979 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45332d00-0bec-4379-8673-81821ba2f220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:51.991518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-45332d00-0bec-4379-8673-81821ba2f220 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:51.992866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-45332d00-0bec-4379-8673-81821ba2f220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:52.067036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1076/2137] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:52.080183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:52.083173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:52.160796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054cfa4-d3de-4242-a400-c64f967b5c8a admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:52.171231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b90c8-b901-4143-ba9c-046413753330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:52.171995 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b90c8-b901-4143-ba9c-046413753330 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:52.173646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b90c8-b901-4143-ba9c-046413753330 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:52.174411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b90c8-b901-4143-ba9c-046413753330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:52.189247 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b90c8-b901-4143-ba9c-046413753330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:52.195523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b90c8-b901-4143-ba9c-046413753330 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:52.211818 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1062/2138] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:52.571518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45332d00-0bec-4379-8673-81821ba2f220 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:52.573085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-45332d00-0bec-4379-8673-81821ba2f220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:52.593392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45332d00-0bec-4379-8673-81821ba2f220 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:52.599496 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:52.599769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:52.600187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:52.600734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] PATH_INFO: `/v3/projects/36dbbca5eec64d4f83d4d1fc8dc7bdf3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:52.602959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=36dbbca5eec64d4f83d4d1fc8dc7bdf3)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:52.606250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:52.664945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:52.687115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36dbbca5eec64d4f83d4d1fc8dc7bdf3', 'request_id': 'req-427e2301-55f9-40d4-8ce6-baa1e28e5aff'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:52.717564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36dbbca5eec64d4f83d4d1fc8dc7bdf3', 'request_id': 'req-427e2301-55f9-40d4-8ce6-baa1e28e5aff'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:52.737579 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36dbbca5eec64d4f83d4d1fc8dc7bdf3', 'request_id': 'req-427e2301-55f9-40d4-8ce6-baa1e28e5aff'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:52.751485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] The token cache is being invalidate because project 36dbbca5eec64d4f83d4d1fc8dc7bdf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:52.751784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:52.752684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-427e2301-55f9-40d4-8ce6-baa1e28e5aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:52.755140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1077/2139] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:52 2022] DELETE /identity/v3/projects/36dbbca5eec64d4f83d4d1fc8dc7bdf3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:52.760669 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e14f16-8740-4797-8b1e-6bd25f6c51d9 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:52.761117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e14f16-8740-4797-8b1e-6bd25f6c51d9 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:52.761619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e14f16-8740-4797-8b1e-6bd25f6c51d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:52.790912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-00e14f16-8740-4797-8b1e-6bd25f6c51d9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:52.791642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-00e14f16-8740-4797-8b1e-6bd25f6c51d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:52.882474 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1063/2140] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:52.890259 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a15e90-89d6-4050-9ec5-2553e36ed48b None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:52.890560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a15e90-89d6-4050-9ec5-2553e36ed48b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:52.890863 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a15e90-89d6-4050-9ec5-2553e36ed48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:52.905526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e14f16-8740-4797-8b1e-6bd25f6c51d9 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:52.907392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-00e14f16-8740-4797-8b1e-6bd25f6c51d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:52.918487 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-70a15e90-89d6-4050-9ec5-2553e36ed48b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:52.919898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-70a15e90-89d6-4050-9ec5-2553e36ed48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:52.920629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e14f16-8740-4797-8b1e-6bd25f6c51d9 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:52.926175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:52.926812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:52.927092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:52.927368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] PATH_INFO: `/v3/users/9fc72f6965bb4814a5d85bd04769e4cb` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:52.928530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fc72f6965bb4814a5d85bd04769e4cb)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:52.928941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1078/2141] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:52.933294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a15e90-89d6-4050-9ec5-2553e36ed48b None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:52.933787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:52.943516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a15e90-89d6-4050-9ec5-2553e36ed48b None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:52.953697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848a120-adb8-4e9e-8246-dfb771017293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:52.954324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848a120-adb8-4e9e-8246-dfb771017293 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:52.954601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848a120-adb8-4e9e-8246-dfb771017293 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:52.954875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848a120-adb8-4e9e-8246-dfb771017293 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:52.955605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848a120-adb8-4e9e-8246-dfb771017293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:52.955914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848a120-adb8-4e9e-8246-dfb771017293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:52.957571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848a120-adb8-4e9e-8246-dfb771017293 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:52.974361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1079/2142] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:47:52.979133 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848a120-adb8-4e9e-8246-dfb771017293 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:52.993183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848a120-adb8-4e9e-8246-dfb771017293 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:52.993881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fc72f6965bb4814a5d85bd04769e4cb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:52.997967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024b944-09dd-4d85-ac5d-c5b3263684dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:52.998574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024b944-09dd-4d85-ac5d-c5b3263684dd admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:52.998855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024b944-09dd-4d85-ac5d-c5b3263684dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:52.999127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024b944-09dd-4d85-ac5d-c5b3263684dd admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:52.999779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024b944-09dd-4d85-ac5d-c5b3263684dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.003629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024b944-09dd-4d85-ac5d-c5b3263684dd admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.004007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fc72f6965bb4814a5d85bd04769e4cb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:53.004846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fc72f6965bb4814a5d85bd04769e4cb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:53.005527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fc72f6965bb4814a5d85bd04769e4cb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:53.016204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fc72f6965bb4814a5d85bd04769e4cb'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:53.040318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1064/2143] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:52 2022] DELETE /identity/v3/users/9fc72f6965bb4814a5d85bd04769e4cb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:53.042439 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1080/2144] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:52 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:53.046868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.049041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:53.055911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:53.136416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed5ecc-f654-493c-8b89-9023026eaf43 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.198875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae7b50-a3b0-48d0-b246-b91144f787f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.199618 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae7b50-a3b0-48d0-b246-b91144f787f1 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.199901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae7b50-a3b0-48d0-b246-b91144f787f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.200176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae7b50-a3b0-48d0-b246-b91144f787f1 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.200842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae7b50-a3b0-48d0-b246-b91144f787f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.202753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae7b50-a3b0-48d0-b246-b91144f787f1 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.227997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1065/2145] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:53 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:53.231946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024b944-09dd-4d85-ac5d-c5b3263684dd admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.239576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae7b50-a3b0-48d0-b246-b91144f787f1 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.241314 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-acae7b50-a3b0-48d0-b246-b91144f787f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:53.248490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024b944-09dd-4d85-ac5d-c5b3263684dd admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.253796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae7b50-a3b0-48d0-b246-b91144f787f1 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.257073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7081340-8135-41a4-8074-4370187ed6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.257787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7081340-8135-41a4-8074-4370187ed6cd admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.258068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7081340-8135-41a4-8074-4370187ed6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.258340 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7081340-8135-41a4-8074-4370187ed6cd admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.259026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7081340-8135-41a4-8074-4370187ed6cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.260702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7081340-8135-41a4-8074-4370187ed6cd admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.264913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.266243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.266563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.266963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] PATH_INFO: `/v3/projects/afd56c15beff4515a44799ee7ec083a3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.268459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd56c15beff4515a44799ee7ec083a3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.270394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.274383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1081/2146] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:53.278873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7081340-8135-41a4-8074-4370187ed6cd admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.289778 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7081340-8135-41a4-8074-4370187ed6cd admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.294351 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80818bef-0ea5-467f-bac9-bd59209c20b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.294996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80818bef-0ea5-467f-bac9-bd59209c20b4 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.295197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80818bef-0ea5-467f-bac9-bd59209c20b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.295466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80818bef-0ea5-467f-bac9-bd59209c20b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.296091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80818bef-0ea5-467f-bac9-bd59209c20b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.298018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80818bef-0ea5-467f-bac9-bd59209c20b4 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.309204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1082/2147] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:53.313378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80818bef-0ea5-467f-bac9-bd59209c20b4 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.320564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd56c15beff4515a44799ee7ec083a3', 'request_id': 'req-bb4478ff-c352-485e-9d1c-18f62ea76ced'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:53.323816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80818bef-0ea5-467f-bac9-bd59209c20b4 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.326404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd56c15beff4515a44799ee7ec083a3', 'request_id': 'req-bb4478ff-c352-485e-9d1c-18f62ea76ced'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:53.334794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd56c15beff4515a44799ee7ec083a3', 'request_id': 'req-bb4478ff-c352-485e-9d1c-18f62ea76ced'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:53.356927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] The token cache is being invalidate because project afd56c15beff4515a44799ee7ec083a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:53.357221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:53.358135 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-bb4478ff-c352-485e-9d1c-18f62ea76ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:53.360593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1066/2148] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:53 2022] DELETE /identity/v3/projects/afd56c15beff4515a44799ee7ec083a3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:53.365279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8ff8a-5ad3-4304-ba46-5e69e855104a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.365436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8ff8a-5ad3-4304-ba46-5e69e855104a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.365692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8ff8a-5ad3-4304-ba46-5e69e855104a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.384918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-03f8ff8a-5ad3-4304-ba46-5e69e855104a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:53.386099 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-03f8ff8a-5ad3-4304-ba46-5e69e855104a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:53.390249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c87e95-4b3e-4033-ab8f-5cb92e7911e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.391477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c87e95-4b3e-4033-ab8f-5cb92e7911e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.391759 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c87e95-4b3e-4033-ab8f-5cb92e7911e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.392130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c87e95-4b3e-4033-ab8f-5cb92e7911e5 admin admin] PATH_INFO: `/v3/projects/0b0fdc0bfb304b718f26762c19a40d52/users/2a294753ac24474d867ab0ce38602da6/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.394494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1067/2149] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:53.402230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee8c2d-ceb2-45fa-b145-01c1a6036544 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.402512 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee8c2d-ceb2-45fa-b145-01c1a6036544 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.402788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee8c2d-ceb2-45fa-b145-01c1a6036544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.409376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c87e95-4b3e-4033-ab8f-5cb92e7911e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b0fdc0bfb304b718f26762c19a40d52, user_id=2a294753ac24474d867ab0ce38602da6, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.412784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c87e95-4b3e-4033-ab8f-5cb92e7911e5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.423379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-51ee8c2d-ceb2-45fa-b145-01c1a6036544 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:53.424586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-51ee8c2d-ceb2-45fa-b145-01c1a6036544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:53.426852 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1083/2150] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:53 2022] PUT /identity/v3/projects/0b0fdc0bfb304b718f26762c19a40d52/users/2a294753ac24474d867ab0ce38602da6/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:53.436749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:53.436998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cda71-4d96-4068-8cdc-360923d8b276 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.437459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cda71-4d96-4068-8cdc-360923d8b276 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.437727 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cda71-4d96-4068-8cdc-360923d8b276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.468588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-708cda71-4d96-4068-8cdc-360923d8b276 None None] MFA Rules not processed for user `2a294753ac24474d867ab0ce38602da6`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:53.470438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-708cda71-4d96-4068-8cdc-360923d8b276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:53.492181 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:53.496941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1068/2151] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:53.501408 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee8c2d-ceb2-45fa-b145-01c1a6036544 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.511210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee8c2d-ceb2-45fa-b145-01c1a6036544 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.516632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b448-994b-430b-88f0-6a7343975389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.517539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b448-994b-430b-88f0-6a7343975389 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.517539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b448-994b-430b-88f0-6a7343975389 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.517911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b448-994b-430b-88f0-6a7343975389 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.518568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567b448-994b-430b-88f0-6a7343975389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:53.518876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567b448-994b-430b-88f0-6a7343975389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.520522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567b448-994b-430b-88f0-6a7343975389 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.534940 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1069/2152] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:47:53.541419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b448-994b-430b-88f0-6a7343975389 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.549326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1084/2153] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:53.551706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b448-994b-430b-88f0-6a7343975389 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.556647 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957c858-9426-436b-952e-ec013f5c19df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.557283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957c858-9426-436b-952e-ec013f5c19df admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.557563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957c858-9426-436b-952e-ec013f5c19df admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.557836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957c858-9426-436b-952e-ec013f5c19df admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.558469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957c858-9426-436b-952e-ec013f5c19df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.560662 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957c858-9426-436b-952e-ec013f5c19df admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.560843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cda71-4d96-4068-8cdc-360923d8b276 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.562172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-708cda71-4d96-4068-8cdc-360923d8b276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:53.592085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1070/2154] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:53 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:53.597611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957c858-9426-436b-952e-ec013f5c19df admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.642234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cda71-4d96-4068-8cdc-360923d8b276 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.669819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957c858-9426-436b-952e-ec013f5c19df admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.676014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54472da8-04b9-46f8-96cb-8685046de7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.676752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54472da8-04b9-46f8-96cb-8685046de7ee admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.677095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54472da8-04b9-46f8-96cb-8685046de7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.677443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54472da8-04b9-46f8-96cb-8685046de7ee admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.678155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54472da8-04b9-46f8-96cb-8685046de7ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.680151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54472da8-04b9-46f8-96cb-8685046de7ee admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.706765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1071/2155] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:53 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:53.710456 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54472da8-04b9-46f8-96cb-8685046de7ee admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.722757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54472da8-04b9-46f8-96cb-8685046de7ee admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.727521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a97ce1-bafd-4b34-acf1-fa0eb8d9989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.728121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a97ce1-bafd-4b34-acf1-fa0eb8d9989d admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.728429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8cb2a-a333-4f0e-9fe2-f1eda35a7930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.728901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a97ce1-bafd-4b34-acf1-fa0eb8d9989d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.729285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a97ce1-bafd-4b34-acf1-fa0eb8d9989d admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.729398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8cb2a-a333-4f0e-9fe2-f1eda35a7930 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.729615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8cb2a-a333-4f0e-9fe2-f1eda35a7930 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.729939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a97ce1-bafd-4b34-acf1-fa0eb8d9989d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.730038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8cb2a-a333-4f0e-9fe2-f1eda35a7930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.732145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a97ce1-bafd-4b34-acf1-fa0eb8d9989d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.746179 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1072/2156] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:53.750703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a97ce1-bafd-4b34-acf1-fa0eb8d9989d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.760788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a97ce1-bafd-4b34-acf1-fa0eb8d9989d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.765617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b5a2-4b46-4dd6-a165-7405a69839bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.767059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b5a2-4b46-4dd6-a165-7405a69839bd admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.767346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b5a2-4b46-4dd6-a165-7405a69839bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.767631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b5a2-4b46-4dd6-a165-7405a69839bd admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.768279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030b5a2-4b46-4dd6-a165-7405a69839bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.769913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030b5a2-4b46-4dd6-a165-7405a69839bd admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.784403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1073/2157] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:53.786951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b5a2-4b46-4dd6-a165-7405a69839bd admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.795153 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8cb2a-a333-4f0e-9fe2-f1eda35a7930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.796771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8cb2a-a333-4f0e-9fe2-f1eda35a7930 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.796962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b5a2-4b46-4dd6-a165-7405a69839bd admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:53.800513 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:53.801576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702691-0727-4bba-b7fa-dfa63963b563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:53.802451 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702691-0727-4bba-b7fa-dfa63963b563 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.802785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702691-0727-4bba-b7fa-dfa63963b563 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.803057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702691-0727-4bba-b7fa-dfa63963b563 admin admin] PATH_INFO: `/v3/projects/e8541cb2c9e34e538f97f03636e36d8d/users/067ece9f211e478fb1a6e3ffc5e72690/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.807087 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1085/2158] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:47:53.819829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c702691-0727-4bba-b7fa-dfa63963b563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8541cb2c9e34e538f97f03636e36d8d, user_id=067ece9f211e478fb1a6e3ffc5e72690, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:53.822562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c702691-0727-4bba-b7fa-dfa63963b563 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:53.837310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1074/2159] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:53 2022] PUT /identity/v3/projects/e8541cb2c9e34e538f97f03636e36d8d/users/067ece9f211e478fb1a6e3ffc5e72690/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:53.845552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc46e9-72da-444d-85d8-f49674aec960 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:53.848375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc46e9-72da-444d-85d8-f49674aec960 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:53.848375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc46e9-72da-444d-85d8-f49674aec960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:53.869743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ddfc46e9-72da-444d-85d8-f49674aec960 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:53.869743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ddfc46e9-72da-444d-85d8-f49674aec960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:53.940943 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1086/2160] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:53.950493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702691-0727-4bba-b7fa-dfa63963b563 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:53.952510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4c702691-0727-4bba-b7fa-dfa63963b563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:53.992572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:54.023811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702691-0727-4bba-b7fa-dfa63963b563 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:54.034938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636235cc-228e-4f00-8722-2215b7b4566e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:54.035556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636235cc-228e-4f00-8722-2215b7b4566e service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:54.035839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636235cc-228e-4f00-8722-2215b7b4566e service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:54.036118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636235cc-228e-4f00-8722-2215b7b4566e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:54.037084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc46e9-72da-444d-85d8-f49674aec960 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:54.038435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ddfc46e9-72da-444d-85d8-f49674aec960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:54.046029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636235cc-228e-4f00-8722-2215b7b4566e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:54.047833 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636235cc-228e-4f00-8722-2215b7b4566e service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:54.050253 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc46e9-72da-444d-85d8-f49674aec960 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:54.056045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:54.056798 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:54.057081 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:54.057355 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] PATH_INFO: `/v3/users/2a294753ac24474d867ab0ce38602da6` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:54.058522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a294753ac24474d867ab0ce38602da6)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:54.058663 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1075/2161] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:47:54.060784 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:54.116464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a294753ac24474d867ab0ce38602da6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.124528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a294753ac24474d867ab0ce38602da6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.125346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a294753ac24474d867ab0ce38602da6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.126129 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a294753ac24474d867ab0ce38602da6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.133849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a294753ac24474d867ab0ce38602da6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.154045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1087/2162] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:54 2022] DELETE /identity/v3/users/2a294753ac24474d867ab0ce38602da6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:54.159201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6e343-44e8-4d74-98e0-88faf192eee3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:54.159468 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6e343-44e8-4d74-98e0-88faf192eee3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:54.159722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6e343-44e8-4d74-98e0-88faf192eee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:54.180582 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-07a6e343-44e8-4d74-98e0-88faf192eee3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:54.190575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-07a6e343-44e8-4d74-98e0-88faf192eee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:54.261847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1076/2163] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:54.271762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:54.273252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:54.336767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43995171-0e8e-46d4-86a2-e3c2329f4839 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:54.340477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99a81a-1015-4295-b15b-8113bcb120b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:54.341141 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99a81a-1015-4295-b15b-8113bcb120b9 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:54.341475 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99a81a-1015-4295-b15b-8113bcb120b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:54.341802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99a81a-1015-4295-b15b-8113bcb120b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:54.353523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99a81a-1015-4295-b15b-8113bcb120b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:54.355544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99a81a-1015-4295-b15b-8113bcb120b9 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:54.366373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1088/2164] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:54.697769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6e343-44e8-4d74-98e0-88faf192eee3 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:54.699306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-07a6e343-44e8-4d74-98e0-88faf192eee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:54.714079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6e343-44e8-4d74-98e0-88faf192eee3 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:54.720093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:54.720915 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:54.721335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:54.721772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] PATH_INFO: `/v3/projects/0b0fdc0bfb304b718f26762c19a40d52` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:54.723098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b0fdc0bfb304b718f26762c19a40d52)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:54.725556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:54.768479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db7ee-ee55-40c0-844a-5f2faf9bd548 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:54.768774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db7ee-ee55-40c0-844a-5f2faf9bd548 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:54.769051 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db7ee-ee55-40c0-844a-5f2faf9bd548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:54.798537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:54.819263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b0fdc0bfb304b718f26762c19a40d52', 'request_id': 'req-00bde379-633c-4cd1-9926-3d4cc57ec5bd'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.822950 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-a17db7ee-ee55-40c0-844a-5f2faf9bd548 None None] MFA Rules not processed for user `067ece9f211e478fb1a6e3ffc5e72690`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:54.824595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a17db7ee-ee55-40c0-844a-5f2faf9bd548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:54.826752 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b0fdc0bfb304b718f26762c19a40d52', 'request_id': 'req-00bde379-633c-4cd1-9926-3d4cc57ec5bd'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.837511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b0fdc0bfb304b718f26762c19a40d52', 'request_id': 'req-00bde379-633c-4cd1-9926-3d4cc57ec5bd'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.853516 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] The token cache is being invalidate because project 0b0fdc0bfb304b718f26762c19a40d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:54.854106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.854769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-00bde379-633c-4cd1-9926-3d4cc57ec5bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:54.856450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1077/2165] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:54 2022] DELETE /identity/v3/projects/0b0fdc0bfb304b718f26762c19a40d52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:54.862002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22442c47-2070-473e-a80f-41590d9c2dc9 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:54.862759 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22442c47-2070-473e-a80f-41590d9c2dc9 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:54.863293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22442c47-2070-473e-a80f-41590d9c2dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:54.884619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-22442c47-2070-473e-a80f-41590d9c2dc9 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:54.885963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-22442c47-2070-473e-a80f-41590d9c2dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:54.908743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1089/2166] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:54.913264 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17db7ee-ee55-40c0-844a-5f2faf9bd548 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:54.914503 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a17db7ee-ee55-40c0-844a-5f2faf9bd548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:54.951019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:54.973642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1078/2167] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:54.982780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b489f-e6e0-4a69-aeff-0cb63360f278 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:54.983279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b489f-e6e0-4a69-aeff-0cb63360f278 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:54.983555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b489f-e6e0-4a69-aeff-0cb63360f278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.005784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-1d6b489f-e6e0-4a69-aeff-0cb63360f278 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:55.006323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17db7ee-ee55-40c0-844a-5f2faf9bd548 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.006998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1d6b489f-e6e0-4a69-aeff-0cb63360f278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:55.011726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee385a06-cf47-4d82-9564-957ef54c327f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.012326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee385a06-cf47-4d82-9564-957ef54c327f admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.012636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee385a06-cf47-4d82-9564-957ef54c327f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.012905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee385a06-cf47-4d82-9564-957ef54c327f admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.013536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee385a06-cf47-4d82-9564-957ef54c327f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.015455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee385a06-cf47-4d82-9564-957ef54c327f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.016795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1079/2168] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:55.021800 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b489f-e6e0-4a69-aeff-0cb63360f278 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.030905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b489f-e6e0-4a69-aeff-0cb63360f278 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.036156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156269f1-7bee-4b51-b6fe-19fe1d1f1096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.036904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156269f1-7bee-4b51-b6fe-19fe1d1f1096 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.037182 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156269f1-7bee-4b51-b6fe-19fe1d1f1096 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.037491 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156269f1-7bee-4b51-b6fe-19fe1d1f1096 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.038109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156269f1-7bee-4b51-b6fe-19fe1d1f1096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:55.038407 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156269f1-7bee-4b51-b6fe-19fe1d1f1096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.040036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156269f1-7bee-4b51-b6fe-19fe1d1f1096 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.049615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1090/2169] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:54 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:55.053261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1080/2170] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:47:55.053999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee385a06-cf47-4d82-9564-957ef54c327f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.057293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156269f1-7bee-4b51-b6fe-19fe1d1f1096 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.067114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156269f1-7bee-4b51-b6fe-19fe1d1f1096 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.121462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee385a06-cf47-4d82-9564-957ef54c327f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.126662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b035d-52ff-4720-b1b6-041e7932358e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.127263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b035d-52ff-4720-b1b6-041e7932358e admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.127566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b035d-52ff-4720-b1b6-041e7932358e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.127842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b035d-52ff-4720-b1b6-041e7932358e admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.128509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b035d-52ff-4720-b1b6-041e7932358e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.130456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b035d-52ff-4720-b1b6-041e7932358e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.130791 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6a94f-55d9-4ac1-8248-66e477dfa7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.131400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6a94f-55d9-4ac1-8248-66e477dfa7c9 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.131686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6a94f-55d9-4ac1-8248-66e477dfa7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.131949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6a94f-55d9-4ac1-8248-66e477dfa7c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.132609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6a94f-55d9-4ac1-8248-66e477dfa7c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.134487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6a94f-55d9-4ac1-8248-66e477dfa7c9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.155650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1091/2171] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:55 2022] POST /identity/v3/users => generated 309 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:55.159655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b035d-52ff-4720-b1b6-041e7932358e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.162868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1081/2172] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:55 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:55.167140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6a94f-55d9-4ac1-8248-66e477dfa7c9 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.169317 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b035d-52ff-4720-b1b6-041e7932358e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.182868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:55.229272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998da5cc-9829-4781-bf83-e01332cb497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.229783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998da5cc-9829-4781-bf83-e01332cb497a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.230064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998da5cc-9829-4781-bf83-e01332cb497a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.230331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998da5cc-9829-4781-bf83-e01332cb497a admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.230965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998da5cc-9829-4781-bf83-e01332cb497a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.232587 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998da5cc-9829-4781-bf83-e01332cb497a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.239478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6a94f-55d9-4ac1-8248-66e477dfa7c9 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.243590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1092/2173] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:55.244928 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff1069-5ecb-4fda-b060-b97787c5609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.245537 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff1069-5ecb-4fda-b060-b97787c5609e admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.245837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff1069-5ecb-4fda-b060-b97787c5609e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.246093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff1069-5ecb-4fda-b060-b97787c5609e admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.246820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff1069-5ecb-4fda-b060-b97787c5609e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.247533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998da5cc-9829-4781-bf83-e01332cb497a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.248784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff1069-5ecb-4fda-b060-b97787c5609e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.257205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998da5cc-9829-4781-bf83-e01332cb497a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.262628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150886c2-9966-4846-bd34-1b76f47487a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.263304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150886c2-9966-4846-bd34-1b76f47487a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.263554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150886c2-9966-4846-bd34-1b76f47487a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.263825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150886c2-9966-4846-bd34-1b76f47487a6 admin admin] PATH_INFO: `/v3/projects/4bd2f5b4ea034eb4ab7b66d662ef0d7e/users/3afdcee3942d419199927a57323b0ea7/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.272951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1082/2174] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:55 2022] POST /identity/v3/users => generated 282 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:55.276998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff1069-5ecb-4fda-b060-b97787c5609e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.279977 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150886c2-9966-4846-bd34-1b76f47487a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd2f5b4ea034eb4ab7b66d662ef0d7e, user_id=3afdcee3942d419199927a57323b0ea7, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.283211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150886c2-9966-4846-bd34-1b76f47487a6 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.287947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff1069-5ecb-4fda-b060-b97787c5609e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.293031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6728792-a9e3-4417-93b4-f3617c7da901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.293638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6728792-a9e3-4417-93b4-f3617c7da901 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.293947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6728792-a9e3-4417-93b4-f3617c7da901 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.294193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6728792-a9e3-4417-93b4-f3617c7da901 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.294830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6728792-a9e3-4417-93b4-f3617c7da901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.296435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6728792-a9e3-4417-93b4-f3617c7da901 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.297624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1093/2175] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:55 2022] PUT /identity/v3/projects/4bd2f5b4ea034eb4ab7b66d662ef0d7e/users/3afdcee3942d419199927a57323b0ea7/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:55.301821 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150886c2-9966-4846-bd34-1b76f47487a6 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.307493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1083/2176] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:55.311126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6728792-a9e3-4417-93b4-f3617c7da901 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.321106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6728792-a9e3-4417-93b4-f3617c7da901 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.361428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:55.370520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150886c2-9966-4846-bd34-1b76f47487a6 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.375846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff2248-c9cb-4a97-b2ed-cdf5eec8ea03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.376485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff2248-c9cb-4a97-b2ed-cdf5eec8ea03 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.376767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff2248-c9cb-4a97-b2ed-cdf5eec8ea03 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.377036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff2248-c9cb-4a97-b2ed-cdf5eec8ea03 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.377675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff2248-c9cb-4a97-b2ed-cdf5eec8ea03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.379291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff2248-c9cb-4a97-b2ed-cdf5eec8ea03 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.386269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329e359-1fee-46b3-88f9-1cadf11cbe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.387018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e359-1fee-46b3-88f9-1cadf11cbe7e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.387386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e359-1fee-46b3-88f9-1cadf11cbe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.387666 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e359-1fee-46b3-88f9-1cadf11cbe7e admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.388617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329e359-1fee-46b3-88f9-1cadf11cbe7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.389035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1094/2177] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:55.390776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329e359-1fee-46b3-88f9-1cadf11cbe7e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.393186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff2248-c9cb-4a97-b2ed-cdf5eec8ea03 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.402773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff2248-c9cb-4a97-b2ed-cdf5eec8ea03 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.403345 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1084/2178] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:55.407646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329e359-1fee-46b3-88f9-1cadf11cbe7e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.410689 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6e682-fa0e-4698-b850-c1557287fae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.411337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6e682-fa0e-4698-b850-c1557287fae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.411614 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6e682-fa0e-4698-b850-c1557287fae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.411885 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6e682-fa0e-4698-b850-c1557287fae2 admin admin] PATH_INFO: `/v3/projects/4bd2f5b4ea034eb4ab7b66d662ef0d7e/users/3afdcee3942d419199927a57323b0ea7/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.413805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6e682-fa0e-4698-b850-c1557287fae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd2f5b4ea034eb4ab7b66d662ef0d7e, user_id=3afdcee3942d419199927a57323b0ea7, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.416464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6e682-fa0e-4698-b850-c1557287fae2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.417348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329e359-1fee-46b3-88f9-1cadf11cbe7e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.422706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db271f62-4e82-42b5-9065-f83923b139fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.423395 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db271f62-4e82-42b5-9065-f83923b139fb admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.423671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db271f62-4e82-42b5-9065-f83923b139fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.423943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db271f62-4e82-42b5-9065-f83923b139fb admin admin] PATH_INFO: `/v3/projects/a2047559e3bd403b88628d5eebf0a51a/users/235a4c7901bc49f6a2081fe3f600bbca/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.428086 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1095/2179] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:55 2022] PUT /identity/v3/projects/4bd2f5b4ea034eb4ab7b66d662ef0d7e/users/3afdcee3942d419199927a57323b0ea7/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:55.444093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db271f62-4e82-42b5-9065-f83923b139fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2047559e3bd403b88628d5eebf0a51a, user_id=235a4c7901bc49f6a2081fe3f600bbca, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.447541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db271f62-4e82-42b5-9065-f83923b139fb admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.462355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1085/2180] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:55 2022] PUT /identity/v3/projects/a2047559e3bd403b88628d5eebf0a51a/users/235a4c7901bc49f6a2081fe3f600bbca/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:55.473836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd8db7-d1b1-4a3b-9bcc-7824bb9bb8f8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.474375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd8db7-d1b1-4a3b-9bcc-7824bb9bb8f8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.474718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd8db7-d1b1-4a3b-9bcc-7824bb9bb8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.511006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-f2dd8db7-d1b1-4a3b-9bcc-7824bb9bb8f8 None None] MFA Rules not processed for user `235a4c7901bc49f6a2081fe3f600bbca`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:55.512330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-f2dd8db7-d1b1-4a3b-9bcc-7824bb9bb8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:55.587793 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1096/2181] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:55.592757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db271f62-4e82-42b5-9065-f83923b139fb admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.678846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db271f62-4e82-42b5-9065-f83923b139fb admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.683522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd357446-115c-423f-a8b3-293e1cecdd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.684129 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd357446-115c-423f-a8b3-293e1cecdd0d admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.684445 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd357446-115c-423f-a8b3-293e1cecdd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.684724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd357446-115c-423f-a8b3-293e1cecdd0d admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.685365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd357446-115c-423f-a8b3-293e1cecdd0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.687271 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd357446-115c-423f-a8b3-293e1cecdd0d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.723670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1086/2182] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:55 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:55.727884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd8db7-d1b1-4a3b-9bcc-7824bb9bb8f8 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.733603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:55.741070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd8db7-d1b1-4a3b-9bcc-7824bb9bb8f8 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.823230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277db77-a525-4bb9-89e0-7b1ec10fc1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.823230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277db77-a525-4bb9-89e0-7b1ec10fc1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.823544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277db77-a525-4bb9-89e0-7b1ec10fc1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.823865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277db77-a525-4bb9-89e0-7b1ec10fc1a5 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.824854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277db77-a525-4bb9-89e0-7b1ec10fc1a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.827403 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277db77-a525-4bb9-89e0-7b1ec10fc1a5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.857030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1097/2183] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:55 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:47:55.865663 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd357446-115c-423f-a8b3-293e1cecdd0d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.877029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd357446-115c-423f-a8b3-293e1cecdd0d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.882379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e014d-dc35-4a3e-abfd-44df5250105e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.883204 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e014d-dc35-4a3e-abfd-44df5250105e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.883319 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e014d-dc35-4a3e-abfd-44df5250105e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.883528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e014d-dc35-4a3e-abfd-44df5250105e admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.884490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e014d-dc35-4a3e-abfd-44df5250105e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.886140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e014d-dc35-4a3e-abfd-44df5250105e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.903466 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1087/2184] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:55.904495 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277db77-a525-4bb9-89e0-7b1ec10fc1a5 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.916281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277db77-a525-4bb9-89e0-7b1ec10fc1a5 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:55.920981 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23d199-0a44-4388-8694-c4614ac79131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:55.922022 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23d199-0a44-4388-8694-c4614ac79131 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:55.922465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23d199-0a44-4388-8694-c4614ac79131 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:55.922750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23d199-0a44-4388-8694-c4614ac79131 admin admin] PATH_INFO: `/v3/projects/e0da030e8efb46db839dac059eccd037/users/b1790c9711e8493b88ea562d908047a5/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:55.938773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23d199-0a44-4388-8694-c4614ac79131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0da030e8efb46db839dac059eccd037, user_id=b1790c9711e8493b88ea562d908047a5, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:55.941814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23d199-0a44-4388-8694-c4614ac79131 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:55.955292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1098/2185] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:55 2022] PUT /identity/v3/projects/e0da030e8efb46db839dac059eccd037/users/b1790c9711e8493b88ea562d908047a5/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:55.959709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e014d-dc35-4a3e-abfd-44df5250105e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:55.970683 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e014d-dc35-4a3e-abfd-44df5250105e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:56.035758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dd4fa-de09-4569-821f-670ed08e502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:56.036661 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dd4fa-de09-4569-821f-670ed08e502f admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:56.036970 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dd4fa-de09-4569-821f-670ed08e502f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:56.037248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dd4fa-de09-4569-821f-670ed08e502f admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:56.037896 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3dd4fa-de09-4569-821f-670ed08e502f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:56.039501 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3dd4fa-de09-4569-821f-670ed08e502f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:56.053538 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1088/2186] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:56.054861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23d199-0a44-4388-8694-c4614ac79131 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:56.066783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23d199-0a44-4388-8694-c4614ac79131 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:56.074197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f2d48-4739-4575-91b9-3372b5845f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:56.075023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f2d48-4739-4575-91b9-3372b5845f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:56.081707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f2d48-4739-4575-91b9-3372b5845f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:56.081707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f2d48-4739-4575-91b9-3372b5845f31 admin admin] PATH_INFO: `/v3/projects/e0da030e8efb46db839dac059eccd037/users/b1790c9711e8493b88ea562d908047a5/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:56.081707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f2d48-4739-4575-91b9-3372b5845f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0da030e8efb46db839dac059eccd037, user_id=b1790c9711e8493b88ea562d908047a5, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:56.082870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f2d48-4739-4575-91b9-3372b5845f31 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:56.095282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1099/2187] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:56 2022] PUT /identity/v3/projects/e0da030e8efb46db839dac059eccd037/users/b1790c9711e8493b88ea562d908047a5/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:56.106002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa04ed-5fc0-4b8b-b862-49dda2816905 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:56.106267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa04ed-5fc0-4b8b-b862-49dda2816905 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:56.106539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa04ed-5fc0-4b8b-b862-49dda2816905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:56.144070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5baa04ed-5fc0-4b8b-b862-49dda2816905 None None] MFA Rules not processed for user `b1790c9711e8493b88ea562d908047a5`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:56.145328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5baa04ed-5fc0-4b8b-b862-49dda2816905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:56.153680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c62a9-af9c-4c28-92db-a93a77efce14 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:56.153966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c62a9-af9c-4c28-92db-a93a77efce14 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:56.154238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c62a9-af9c-4c28-92db-a93a77efce14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:56.155899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:56.188128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ba2c62a9-af9c-4c28-92db-a93a77efce14 None None] MFA Rules not processed for user `3afdcee3942d419199927a57323b0ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:56.189406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ba2c62a9-af9c-4c28-92db-a93a77efce14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:56.247919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1089/2188] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:56.257779 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa04ed-5fc0-4b8b-b862-49dda2816905 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:56.259356 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5baa04ed-5fc0-4b8b-b862-49dda2816905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:56.280891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1100/2189] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:56.335336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa04ed-5fc0-4b8b-b862-49dda2816905 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:56.352440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24156bb9-1401-425c-a758-6952ff389a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:56.353180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156bb9-1401-425c-a758-6952ff389a33 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:56.353859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156bb9-1401-425c-a758-6952ff389a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:56.354703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156bb9-1401-425c-a758-6952ff389a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:56.365598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24156bb9-1401-425c-a758-6952ff389a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:56.367191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24156bb9-1401-425c-a758-6952ff389a33 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:56.378478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1090/2190] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:47:56.923353 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c62a9-af9c-4c28-92db-a93a77efce14 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:56.998158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:57.027518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c62a9-af9c-4c28-92db-a93a77efce14 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:57.040363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:57.041031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:57.041358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:57.041635 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] PATH_INFO: `/v3/users/235a4c7901bc49f6a2081fe3f600bbca` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:57.045201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235a4c7901bc49f6a2081fe3f600bbca)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:57.048523 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:57.109662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235a4c7901bc49f6a2081fe3f600bbca'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.116831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235a4c7901bc49f6a2081fe3f600bbca'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.117513 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235a4c7901bc49f6a2081fe3f600bbca'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.118121 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235a4c7901bc49f6a2081fe3f600bbca'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.126797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235a4c7901bc49f6a2081fe3f600bbca'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.152577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1101/2191] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:56 2022] DELETE /identity/v3/users/235a4c7901bc49f6a2081fe3f600bbca => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:57.157831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d426267-b34f-48e5-abcb-d4e263a13e1f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:57.158317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d426267-b34f-48e5-abcb-d4e263a13e1f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:57.158577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d426267-b34f-48e5-abcb-d4e263a13e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:57.182243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-7d426267-b34f-48e5-abcb-d4e263a13e1f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:57.183586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7d426267-b34f-48e5-abcb-d4e263a13e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:57.273007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:57.274752 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:57.277205 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1091/2192] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:57.348700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff567b9-2bc6-46df-bbda-5b242db08317 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:57.352525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a7e02-d408-425f-b015-5e1fa03610fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:57.353250 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a7e02-d408-425f-b015-5e1fa03610fe service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:57.353872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a7e02-d408-425f-b015-5e1fa03610fe service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:57.354217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a7e02-d408-425f-b015-5e1fa03610fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:57.364270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a7e02-d408-425f-b015-5e1fa03610fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:57.365839 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a7e02-d408-425f-b015-5e1fa03610fe service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:57.376179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1102/2193] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:47:57.510169 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d426267-b34f-48e5-abcb-d4e263a13e1f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:57.511654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7d426267-b34f-48e5-abcb-d4e263a13e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:57.533254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d426267-b34f-48e5-abcb-d4e263a13e1f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:57.540844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:57.541873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:57.542351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:57.542817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] PATH_INFO: `/v3/users/067ece9f211e478fb1a6e3ffc5e72690` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:57.544696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=067ece9f211e478fb1a6e3ffc5e72690)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:57.548445 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:57.600716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:57.607322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '067ece9f211e478fb1a6e3ffc5e72690'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.616695 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '067ece9f211e478fb1a6e3ffc5e72690'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.617508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '067ece9f211e478fb1a6e3ffc5e72690'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.618614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '067ece9f211e478fb1a6e3ffc5e72690'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.627466 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '067ece9f211e478fb1a6e3ffc5e72690'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.647524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1092/2194] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:57 2022] DELETE /identity/v3/users/067ece9f211e478fb1a6e3ffc5e72690 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:57.755605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a7e02-d408-425f-b015-5e1fa03610fe service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:57.757427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1c8a7e02-d408-425f-b015-5e1fa03610fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:57.777286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a7e02-d408-425f-b015-5e1fa03610fe service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:57.840969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:57.842242 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:57.842406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:57.842897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:57.843178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:57.843519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] PATH_INFO: `/v3/projects/a2047559e3bd403b88628d5eebf0a51a` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:57.846002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2047559e3bd403b88628d5eebf0a51a)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:57.848450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:57.851569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:57.868063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f47e5-b17e-461e-b91f-34ac3df048c3 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:57.873818 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:57.874454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:57.874730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:57.875265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] PATH_INFO: `/v3/projects/e8541cb2c9e34e538f97f03636e36d8d` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:57.876768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8541cb2c9e34e538f97f03636e36d8d)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:57.878695 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:57.903888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2047559e3bd403b88628d5eebf0a51a', 'request_id': 'req-64b65981-f6ea-467c-9917-ae0b27d36cb9'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.910549 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2047559e3bd403b88628d5eebf0a51a', 'request_id': 'req-64b65981-f6ea-467c-9917-ae0b27d36cb9'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.917548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2047559e3bd403b88628d5eebf0a51a', 'request_id': 'req-64b65981-f6ea-467c-9917-ae0b27d36cb9'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.931411 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8541cb2c9e34e538f97f03636e36d8d', 'request_id': 'req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.931741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] The token cache is being invalidate because project a2047559e3bd403b88628d5eebf0a51a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:57.932037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.932736 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.934444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1103/2195] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:57 2022] DELETE /identity/v3/projects/a2047559e3bd403b88628d5eebf0a51a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:57.937062 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8541cb2c9e34e538f97f03636e36d8d', 'request_id': 'req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.938477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:57.939896 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:57.944712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8541cb2c9e34e538f97f03636e36d8d', 'request_id': 'req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.956294 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] The token cache is being invalidate because project e8541cb2c9e34e538f97f03636e36d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:57.956611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.957239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:57.958707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1093/2196] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:57 2022] DELETE /identity/v3/projects/e8541cb2c9e34e538f97f03636e36d8d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:57.963479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:57.964915 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:58.011547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b65981-f6ea-467c-9917-ae0b27d36cb9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:58.051849 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd1c0-b5b1-4b8a-80b1-cfc6a76c02ba admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:58.057719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:58.058351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.058632 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.058907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] PATH_INFO: `/v3/users/3afdcee3942d419199927a57323b0ea7` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.060069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3afdcee3942d419199927a57323b0ea7)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:58.062180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:58.086245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:58.086898 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.087183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.087454 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] PATH_INFO: `/v3/users/b1790c9711e8493b88ea562d908047a5` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.088660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1790c9711e8493b88ea562d908047a5)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:58.090757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:58.117346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:58.119704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3afdcee3942d419199927a57323b0ea7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.129176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3afdcee3942d419199927a57323b0ea7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.129944 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3afdcee3942d419199927a57323b0ea7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.130724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3afdcee3942d419199927a57323b0ea7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.138405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3afdcee3942d419199927a57323b0ea7'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.142966 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:58.149707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1790c9711e8493b88ea562d908047a5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.158540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1094/2197] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:57 2022] DELETE /identity/v3/users/3afdcee3942d419199927a57323b0ea7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:58.161624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1790c9711e8493b88ea562d908047a5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.162616 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1790c9711e8493b88ea562d908047a5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.163324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1790c9711e8493b88ea562d908047a5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.170612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1790c9711e8493b88ea562d908047a5'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.189991 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1104/2198] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:57 2022] DELETE /identity/v3/users/b1790c9711e8493b88ea562d908047a5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:58.362042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:58.364090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:58.498332 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616abdd8-358e-4eef-bddd-d52bb55767bb admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:58.507742 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:58.509306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:58.584256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:58.585193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.585193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.585491 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] PATH_INFO: `/v3/projects/4bd2f5b4ea034eb4ab7b66d662ef0d7e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.586651 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd2f5b4ea034eb4ab7b66d662ef0d7e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:58.588706 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:58.598657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134e8ff-0e08-47ea-ad6f-67c5dc2f30d2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:58.604071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:58.604755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.605189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.605489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] PATH_INFO: `/v3/projects/e0da030e8efb46db839dac059eccd037` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.608290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0da030e8efb46db839dac059eccd037)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:58.616027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:58.659847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd2f5b4ea034eb4ab7b66d662ef0d7e', 'request_id': 'req-046b269d-ac15-4176-8c1c-af274bb16f3c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.664197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0da030e8efb46db839dac059eccd037', 'request_id': 'req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.669274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd2f5b4ea034eb4ab7b66d662ef0d7e', 'request_id': 'req-046b269d-ac15-4176-8c1c-af274bb16f3c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.670254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0da030e8efb46db839dac059eccd037', 'request_id': 'req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.680492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0da030e8efb46db839dac059eccd037', 'request_id': 'req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.683954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd2f5b4ea034eb4ab7b66d662ef0d7e', 'request_id': 'req-046b269d-ac15-4176-8c1c-af274bb16f3c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.692945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] The token cache is being invalidate because project e0da030e8efb46db839dac059eccd037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:58.693224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.693877 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6b85ee9d-d6be-48e8-ac2b-4c1363254a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.694969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:58.695361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1105/2199] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:58 2022] DELETE /identity/v3/projects/e0da030e8efb46db839dac059eccd037 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:58.700043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] The token cache is being invalidate because project 4bd2f5b4ea034eb4ab7b66d662ef0d7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:47:58.701099 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd46638-bc51-42e1-ad6c-f63ae4d9f335 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.701453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd46638-bc51-42e1-ad6c-f63ae4d9f335 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.701639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.701800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd46638-bc51-42e1-ad6c-f63ae4d9f335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.704559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-046b269d-ac15-4176-8c1c-af274bb16f3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:58.707063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1095/2200] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:47:58 2022] DELETE /identity/v3/projects/4bd2f5b4ea034eb4ab7b66d662ef0d7e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:47:58.713288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd0386-43f4-4550-a46c-a89fcc798f81 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.715235 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd0386-43f4-4550-a46c-a89fcc798f81 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.715990 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd0386-43f4-4550-a46c-a89fcc798f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.731067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-ecd46638-bc51-42e1-ad6c-f63ae4d9f335 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:58.732281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ecd46638-bc51-42e1-ad6c-f63ae4d9f335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:58.742495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-81cd0386-43f4-4550-a46c-a89fcc798f81 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:58.743836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-81cd0386-43f4-4550-a46c-a89fcc798f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:58.786029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:58.808030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1106/2201] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:58.818201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12bc50-284d-43c5-a302-c821e4b18a53 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.818488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12bc50-284d-43c5-a302-c821e4b18a53 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.818766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12bc50-284d-43c5-a302-c821e4b18a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.836723 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1096/2202] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:58.843613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-6c12bc50-284d-43c5-a302-c821e4b18a53 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:58.845668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6c12bc50-284d-43c5-a302-c821e4b18a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:58.847065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae4914-a116-4c8e-926f-344eb542b316 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.847428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae4914-a116-4c8e-926f-344eb542b316 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.847707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae4914-a116-4c8e-926f-344eb542b316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.855565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1107/2203] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:58.859967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12bc50-284d-43c5-a302-c821e4b18a53 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:58.868796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b6ae4914-a116-4c8e-926f-344eb542b316 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:58.868979 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12bc50-284d-43c5-a302-c821e4b18a53 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:58.869987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b6ae4914-a116-4c8e-926f-344eb542b316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:58.873373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da470e34-2694-482e-a8f6-e3d16d7a88d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:58.873951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da470e34-2694-482e-a8f6-e3d16d7a88d4 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.874229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da470e34-2694-482e-a8f6-e3d16d7a88d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.874498 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da470e34-2694-482e-a8f6-e3d16d7a88d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.875711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da470e34-2694-482e-a8f6-e3d16d7a88d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:58.876003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da470e34-2694-482e-a8f6-e3d16d7a88d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:58.877617 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da470e34-2694-482e-a8f6-e3d16d7a88d4 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:58.878517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1097/2204] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:58.882629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae4914-a116-4c8e-926f-344eb542b316 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:58.889879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1108/2205] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:47:58.894462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae4914-a116-4c8e-926f-344eb542b316 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:58.894604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da470e34-2694-482e-a8f6-e3d16d7a88d4 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:58.900270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a89bd-4b38-4c38-b52b-af27ed70a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:58.900796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a89bd-4b38-4c38-b52b-af27ed70a0ae admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.901077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a89bd-4b38-4c38-b52b-af27ed70a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.902000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a89bd-4b38-4c38-b52b-af27ed70a0ae admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.902000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a89bd-4b38-4c38-b52b-af27ed70a0ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:47:58.902309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a89bd-4b38-4c38-b52b-af27ed70a0ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:58.903963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a89bd-4b38-4c38-b52b-af27ed70a0ae admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:58.907148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da470e34-2694-482e-a8f6-e3d16d7a88d4 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:58.912484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da626922-e1a1-482b-a6dc-761098b047ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:58.913396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da626922-e1a1-482b-a6dc-761098b047ce admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.913396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da626922-e1a1-482b-a6dc-761098b047ce admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.913618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da626922-e1a1-482b-a6dc-761098b047ce admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.914230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da626922-e1a1-482b-a6dc-761098b047ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:58.916463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da626922-e1a1-482b-a6dc-761098b047ce admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:58.920170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1098/2206] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:47:58.924797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a89bd-4b38-4c38-b52b-af27ed70a0ae admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:58.935789 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a89bd-4b38-4c38-b52b-af27ed70a0ae admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:58.942298 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2ba79-0873-44bf-9d52-633fa462dfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:58.943014 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ba79-0873-44bf-9d52-633fa462dfac admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:58.943389 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ba79-0873-44bf-9d52-633fa462dfac admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:58.943732 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ba79-0873-44bf-9d52-633fa462dfac admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:58.944547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2ba79-0873-44bf-9d52-633fa462dfac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:58.946500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2ba79-0873-44bf-9d52-633fa462dfac admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:58.947124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1109/2207] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:58 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:58.951064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da626922-e1a1-482b-a6dc-761098b047ce admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:58.981260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1099/2208] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:58 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:58.982229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2ba79-0873-44bf-9d52-633fa462dfac admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.023693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da626922-e1a1-482b-a6dc-761098b047ce admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.054557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2ba79-0873-44bf-9d52-633fa462dfac admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.063545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50e5bf-6312-4e22-878a-c3043ec8ae43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.064234 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50e5bf-6312-4e22-878a-c3043ec8ae43 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.064473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50e5bf-6312-4e22-878a-c3043ec8ae43 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.064773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50e5bf-6312-4e22-878a-c3043ec8ae43 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.065432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50e5bf-6312-4e22-878a-c3043ec8ae43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.067486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50e5bf-6312-4e22-878a-c3043ec8ae43 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.096328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1100/2209] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:58 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:59.096685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1785ac-c68c-4864-9300-8385fd5ee8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.097285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1785ac-c68c-4864-9300-8385fd5ee8cf admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.097575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1785ac-c68c-4864-9300-8385fd5ee8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.097703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50e5bf-6312-4e22-878a-c3043ec8ae43 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.097960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1785ac-c68c-4864-9300-8385fd5ee8cf admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.098906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1785ac-c68c-4864-9300-8385fd5ee8cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.101319 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1785ac-c68c-4864-9300-8385fd5ee8cf admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.109756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50e5bf-6312-4e22-878a-c3043ec8ae43 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.114913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2189445-bd78-4aea-99d3-44b62b3c1a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.115726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2189445-bd78-4aea-99d3-44b62b3c1a88 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.116140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2189445-bd78-4aea-99d3-44b62b3c1a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.116489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2189445-bd78-4aea-99d3-44b62b3c1a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.117264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2189445-bd78-4aea-99d3-44b62b3c1a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.119048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2189445-bd78-4aea-99d3-44b62b3c1a88 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.129868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1110/2210] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:58 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:59.130985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1785ac-c68c-4864-9300-8385fd5ee8cf admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.131188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1101/2211] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:59.137155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2189445-bd78-4aea-99d3-44b62b3c1a88 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.145126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1785ac-c68c-4864-9300-8385fd5ee8cf admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.147350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2189445-bd78-4aea-99d3-44b62b3c1a88 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.150599 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d24611-592c-40e3-b5c9-23bec15a2cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.151197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d24611-592c-40e3-b5c9-23bec15a2cff admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.151491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d24611-592c-40e3-b5c9-23bec15a2cff admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.151764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d24611-592c-40e3-b5c9-23bec15a2cff admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.152000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924edda-7606-49c6-bd83-2e06524e122e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.152434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d24611-592c-40e3-b5c9-23bec15a2cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.152617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924edda-7606-49c6-bd83-2e06524e122e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.152895 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924edda-7606-49c6-bd83-2e06524e122e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.153163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924edda-7606-49c6-bd83-2e06524e122e admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.153797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924edda-7606-49c6-bd83-2e06524e122e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.154057 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d24611-592c-40e3-b5c9-23bec15a2cff admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.155460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924edda-7606-49c6-bd83-2e06524e122e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.165881 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1111/2212] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:59.169258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1102/2213] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:47:59.169754 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d24611-592c-40e3-b5c9-23bec15a2cff admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.173003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924edda-7606-49c6-bd83-2e06524e122e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.179812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d24611-592c-40e3-b5c9-23bec15a2cff admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.182675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924edda-7606-49c6-bd83-2e06524e122e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.184591 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42e867-0440-4b46-8686-ad441a35ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.185186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42e867-0440-4b46-8686-ad441a35ad79 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.185469 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42e867-0440-4b46-8686-ad441a35ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.185765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42e867-0440-4b46-8686-ad441a35ad79 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.186406 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42e867-0440-4b46-8686-ad441a35ad79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.187851 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc63a25-816d-48d9-852a-ac7d7f16d1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.188526 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc63a25-816d-48d9-852a-ac7d7f16d1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.188739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42e867-0440-4b46-8686-ad441a35ad79 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.188857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc63a25-816d-48d9-852a-ac7d7f16d1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.189067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc63a25-816d-48d9-852a-ac7d7f16d1ff admin admin] PATH_INFO: `/v3/projects/a14f637bebd145039c4c78f6155ff2ee/users/23d0da38ee28429282308913581251b6/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.199151 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1112/2214] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:59.204455 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc63a25-816d-48d9-852a-ac7d7f16d1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14f637bebd145039c4c78f6155ff2ee, user_id=23d0da38ee28429282308913581251b6, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.204621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42e867-0440-4b46-8686-ad441a35ad79 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.208880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc63a25-816d-48d9-852a-ac7d7f16d1ff admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.214641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42e867-0440-4b46-8686-ad441a35ad79 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.221650 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1103/2215] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:59 2022] PUT /identity/v3/projects/a14f637bebd145039c4c78f6155ff2ee/users/23d0da38ee28429282308913581251b6/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:59.232285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95b23b-dc70-4246-8f28-c9d214a6a296 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.232610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95b23b-dc70-4246-8f28-c9d214a6a296 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.232891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95b23b-dc70-4246-8f28-c9d214a6a296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.239809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:59.242316 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:59.270643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-7b95b23b-dc70-4246-8f28-c9d214a6a296 None None] MFA Rules not processed for user `23d0da38ee28429282308913581251b6`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:59.271872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7b95b23b-dc70-4246-8f28-c9d214a6a296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:59.286728 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978f579-d0c2-48f7-9689-4a19d306dc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.287384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978f579-d0c2-48f7-9689-4a19d306dc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.287664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978f579-d0c2-48f7-9689-4a19d306dc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.287945 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978f579-d0c2-48f7-9689-4a19d306dc1b admin admin] PATH_INFO: `/v3/projects/76c10ed1f7274fe8b3df08c0150dcae3/users/a0d8e696ac234d2dafbd82d07a373d9c/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.303445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978f579-d0c2-48f7-9689-4a19d306dc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c10ed1f7274fe8b3df08c0150dcae3, user_id=a0d8e696ac234d2dafbd82d07a373d9c, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.306181 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978f579-d0c2-48f7-9689-4a19d306dc1b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.319547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1113/2216] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:59 2022] PUT /identity/v3/projects/76c10ed1f7274fe8b3df08c0150dcae3/users/a0d8e696ac234d2dafbd82d07a373d9c/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:59.330223 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0fc0f-687b-429d-b40e-4a3c0249ed79 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.330528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0fc0f-687b-429d-b40e-4a3c0249ed79 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.330795 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0fc0f-687b-429d-b40e-4a3c0249ed79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.352265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1104/2217] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:59.361118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95b23b-dc70-4246-8f28-c9d214a6a296 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.362520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7b95b23b-dc70-4246-8f28-c9d214a6a296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:59.369747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-92f0fc0f-687b-429d-b40e-4a3c0249ed79 None None] MFA Rules not processed for user `a0d8e696ac234d2dafbd82d07a373d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:59.370990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-92f0fc0f-687b-429d-b40e-4a3c0249ed79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:59.419258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95b23b-dc70-4246-8f28-c9d214a6a296 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.439012 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1114/2218] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:47:59.443907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0fc0f-687b-429d-b40e-4a3c0249ed79 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.468263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16bca5-6920-44d0-bd42-8803e346d81a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.469255 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16bca5-6920-44d0-bd42-8803e346d81a service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.469540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16bca5-6920-44d0-bd42-8803e346d81a service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.469819 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16bca5-6920-44d0-bd42-8803e346d81a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.512466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0fc0f-687b-429d-b40e-4a3c0249ed79 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.518391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf919357-48ee-47ca-8f2d-3192ee90bff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.519026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf919357-48ee-47ca-8f2d-3192ee90bff3 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.519306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf919357-48ee-47ca-8f2d-3192ee90bff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.519602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf919357-48ee-47ca-8f2d-3192ee90bff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.520224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf919357-48ee-47ca-8f2d-3192ee90bff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.522206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf919357-48ee-47ca-8f2d-3192ee90bff3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.529144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16bca5-6920-44d0-bd42-8803e346d81a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.530591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16bca5-6920-44d0-bd42-8803e346d81a service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.541753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1105/2219] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:47:59.550374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1115/2220] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:47:59 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:59.554176 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16bca5-6920-44d0-bd42-8803e346d81a service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.559617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:59.566085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf919357-48ee-47ca-8f2d-3192ee90bff3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.566252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16bca5-6920-44d0-bd42-8803e346d81a service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.588152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:59.638882 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517849-826a-4447-ad70-18f6c71fd512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.639498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517849-826a-4447-ad70-18f6c71fd512 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.639780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517849-826a-4447-ad70-18f6c71fd512 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.640076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517849-826a-4447-ad70-18f6c71fd512 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.640673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf919357-48ee-47ca-8f2d-3192ee90bff3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.641417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517849-826a-4447-ad70-18f6c71fd512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.643368 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517849-826a-4447-ad70-18f6c71fd512 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.645930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.646633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.646971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.647318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] PATH_INFO: `/v3/users/23d0da38ee28429282308913581251b6` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.648685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d0da38ee28429282308913581251b6)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.650550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.668868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1106/2221] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:47:59 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:47:59.669828 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517849-826a-4447-ad70-18f6c71fd512 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.679898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517849-826a-4447-ad70-18f6c71fd512 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.685046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e498a3c-820b-4384-996b-05d46a48ecf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.685640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e498a3c-820b-4384-996b-05d46a48ecf4 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.685921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e498a3c-820b-4384-996b-05d46a48ecf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.686206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e498a3c-820b-4384-996b-05d46a48ecf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.686849 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e498a3c-820b-4384-996b-05d46a48ecf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.689720 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e498a3c-820b-4384-996b-05d46a48ecf4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.699811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1107/2222] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:59.701631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d0da38ee28429282308913581251b6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:59.705084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e498a3c-820b-4384-996b-05d46a48ecf4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.709594 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d0da38ee28429282308913581251b6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:59.710288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d0da38ee28429282308913581251b6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:59.710921 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d0da38ee28429282308913581251b6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:59.715006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e498a3c-820b-4384-996b-05d46a48ecf4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.718733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6f5610cf-f435-43c5-ad4c-3c8f647c5be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d0da38ee28429282308913581251b6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:47:59.720744 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb8644-e198-4f1c-b5a4-f60f32a603be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.721391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb8644-e198-4f1c-b5a4-f60f32a603be admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.721676 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb8644-e198-4f1c-b5a4-f60f32a603be admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.721952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb8644-e198-4f1c-b5a4-f60f32a603be admin admin] PATH_INFO: `/v3/projects/6319f793b1404a74a4071c6328c9ba77/users/2980ef1f198a44d3acc7555193185594/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.738139 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb8644-e198-4f1c-b5a4-f60f32a603be admin admin] RBAC: Authorizing `identity:create_grant(project_id=6319f793b1404a74a4071c6328c9ba77, user_id=2980ef1f198a44d3acc7555193185594, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.738348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1116/2223] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:47:59 2022] DELETE /identity/v3/users/23d0da38ee28429282308913581251b6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:59.740857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb8644-e198-4f1c-b5a4-f60f32a603be admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.742834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf08dc5-6d77-494b-96f3-a6160fd228af None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.743120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf08dc5-6d77-494b-96f3-a6160fd228af None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.743396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf08dc5-6d77-494b-96f3-a6160fd228af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.754289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1108/2224] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:59 2022] PUT /identity/v3/projects/6319f793b1404a74a4071c6328c9ba77/users/2980ef1f198a44d3acc7555193185594/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:59.758076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb8644-e198-4f1c-b5a4-f60f32a603be admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.759725 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-89eb8644-e198-4f1c-b5a4-f60f32a603be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:59.763208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-caf08dc5-6d77-494b-96f3-a6160fd228af None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:59.764409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-caf08dc5-6d77-494b-96f3-a6160fd228af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:59.825257 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb8644-e198-4f1c-b5a4-f60f32a603be admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.827723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1117/2225] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:47:59.834270 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14756254-930f-40db-949b-b5ed380cfdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.834976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14756254-930f-40db-949b-b5ed380cfdba admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.835290 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14756254-930f-40db-949b-b5ed380cfdba admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.835585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14756254-930f-40db-949b-b5ed380cfdba admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.836155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14756254-930f-40db-949b-b5ed380cfdba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.837852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14756254-930f-40db-949b-b5ed380cfdba admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.838002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf08dc5-6d77-494b-96f3-a6160fd228af None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.839649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-caf08dc5-6d77-494b-96f3-a6160fd228af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:59.849981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1109/2226] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:47:59.854231 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14756254-930f-40db-949b-b5ed380cfdba admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:47:59.865737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14756254-930f-40db-949b-b5ed380cfdba admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.870863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f04da7-7747-4b09-8a0f-14a728fc16c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.871521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f04da7-7747-4b09-8a0f-14a728fc16c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.871805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f04da7-7747-4b09-8a0f-14a728fc16c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.872083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f04da7-7747-4b09-8a0f-14a728fc16c9 admin admin] PATH_INFO: `/v3/projects/6319f793b1404a74a4071c6328c9ba77/users/2980ef1f198a44d3acc7555193185594/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.874082 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f04da7-7747-4b09-8a0f-14a728fc16c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6319f793b1404a74a4071c6328c9ba77, user_id=2980ef1f198a44d3acc7555193185594, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:47:59.876843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f04da7-7747-4b09-8a0f-14a728fc16c9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:47:59.886857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:47:59.894579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1110/2227] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:47:59 2022] PUT /identity/v3/projects/6319f793b1404a74a4071c6328c9ba77/users/2980ef1f198a44d3acc7555193185594/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:47:59.899192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c6816-737e-4fac-b305-1ee4009c77fd None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.900210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c6816-737e-4fac-b305-1ee4009c77fd None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.900627 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c6816-737e-4fac-b305-1ee4009c77fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.912602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf08dc5-6d77-494b-96f3-a6160fd228af None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:47:59.941172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-462c6816-737e-4fac-b305-1ee4009c77fd None None] MFA Rules not processed for user `2980ef1f198a44d3acc7555193185594`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:47:59.942599 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-462c6816-737e-4fac-b305-1ee4009c77fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:47:59.967533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11732e-d75d-4724-a783-5999c3cbdcbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:47:59.968159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11732e-d75d-4724-a783-5999c3cbdcbd service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:47:59.969041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11732e-d75d-4724-a783-5999c3cbdcbd service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:47:59.969341 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11732e-d75d-4724-a783-5999c3cbdcbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:47:59.978492 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:00.029511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1111/2228] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:00.042210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c6816-737e-4fac-b305-1ee4009c77fd None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:00.044048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-462c6816-737e-4fac-b305-1ee4009c77fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:00.045529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11732e-d75d-4724-a783-5999c3cbdcbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:00.047127 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11732e-d75d-4724-a783-5999c3cbdcbd service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:00.057917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1118/2229] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:00.128553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c6816-737e-4fac-b305-1ee4009c77fd None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:00.137157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819caaf-7e31-417f-9743-0d92b472c1a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:00.137754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819caaf-7e31-417f-9743-0d92b472c1a4 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:00.141246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819caaf-7e31-417f-9743-0d92b472c1a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:00.141723 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819caaf-7e31-417f-9743-0d92b472c1a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:00.156524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819caaf-7e31-417f-9743-0d92b472c1a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:00.158546 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819caaf-7e31-417f-9743-0d92b472c1a4 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:00.169958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1112/2230] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:00.229349 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11732e-d75d-4724-a783-5999c3cbdcbd service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:00.230948 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9f11732e-d75d-4724-a783-5999c3cbdcbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:00.244647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11732e-d75d-4724-a783-5999c3cbdcbd service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:00.248596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fd193-692d-40d6-8d5c-ae8c1bfe560d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:00.249206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fd193-692d-40d6-8d5c-ae8c1bfe560d service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:00.249483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fd193-692d-40d6-8d5c-ae8c1bfe560d service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:00.250336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fd193-692d-40d6-8d5c-ae8c1bfe560d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:00.252206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-512fd193-692d-40d6-8d5c-ae8c1bfe560d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:00.314895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fd193-692d-40d6-8d5c-ae8c1bfe560d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:00.316331 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fd193-692d-40d6-8d5c-ae8c1bfe560d service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:00.326026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1119/2231] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:00.651419 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819caaf-7e31-417f-9743-0d92b472c1a4 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:00.652889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1819caaf-7e31-417f-9743-0d92b472c1a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:00.667710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819caaf-7e31-417f-9743-0d92b472c1a4 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:00.673237 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:00.674181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:00.674804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:00.675328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] PATH_INFO: `/v3/projects/a14f637bebd145039c4c78f6155ff2ee` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:00.676812 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14f637bebd145039c4c78f6155ff2ee)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:00.679039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:00.729249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14f637bebd145039c4c78f6155ff2ee', 'request_id': 'req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:00.735981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14f637bebd145039c4c78f6155ff2ee', 'request_id': 'req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:00.742621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14f637bebd145039c4c78f6155ff2ee', 'request_id': 'req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:00.753704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] The token cache is being invalidate because project a14f637bebd145039c4c78f6155ff2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:00.753992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:00.754614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-5b2bede9-faaa-4740-aaa8-2cb7c3721c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:00.756069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1113/2232] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:00 2022] DELETE /identity/v3/projects/a14f637bebd145039c4c78f6155ff2ee => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:00.760921 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43164f51-b0b8-4f84-b381-1cb53e7243f0 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:00.761206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43164f51-b0b8-4f84-b381-1cb53e7243f0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:00.761479 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43164f51-b0b8-4f84-b381-1cb53e7243f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:00.781016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-43164f51-b0b8-4f84-b381-1cb53e7243f0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:00.782182 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-43164f51-b0b8-4f84-b381-1cb53e7243f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:00.821711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:00.841225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1120/2233] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:00.850118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32baa5fc-4fac-4bc6-a639-a4b1e913708e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:00.850403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32baa5fc-4fac-4bc6-a639-a4b1e913708e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:00.850681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32baa5fc-4fac-4bc6-a639-a4b1e913708e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:00.870711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-32baa5fc-4fac-4bc6-a639-a4b1e913708e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:00.871825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-32baa5fc-4fac-4bc6-a639-a4b1e913708e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:00.880521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1114/2234] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:00.884735 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43164f51-b0b8-4f84-b381-1cb53e7243f0 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:00.894958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43164f51-b0b8-4f84-b381-1cb53e7243f0 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:00.899452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84b151-71f8-4f20-b2cc-8cb05461e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:00.900042 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84b151-71f8-4f20-b2cc-8cb05461e611 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:00.900322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84b151-71f8-4f20-b2cc-8cb05461e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:00.900639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84b151-71f8-4f20-b2cc-8cb05461e611 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:00.901290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84b151-71f8-4f20-b2cc-8cb05461e611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:00.901595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84b151-71f8-4f20-b2cc-8cb05461e611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:00.903185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84b151-71f8-4f20-b2cc-8cb05461e611 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:00.915102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1121/2235] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:48:00.919236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32baa5fc-4fac-4bc6-a639-a4b1e913708e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:00.928976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32baa5fc-4fac-4bc6-a639-a4b1e913708e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:00.933707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32435bc-67f1-4912-8370-3bf7d8893fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:00.934299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32435bc-67f1-4912-8370-3bf7d8893fde admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:00.934580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32435bc-67f1-4912-8370-3bf7d8893fde admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:00.934857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32435bc-67f1-4912-8370-3bf7d8893fde admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:00.935556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32435bc-67f1-4912-8370-3bf7d8893fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:00.938155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32435bc-67f1-4912-8370-3bf7d8893fde admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:00.964908 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1115/2236] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:00 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:00.969565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84b151-71f8-4f20-b2cc-8cb05461e611 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:00.979572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84b151-71f8-4f20-b2cc-8cb05461e611 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:01.035309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27eead-79de-4129-b2ad-064fca2c8f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:01.035912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27eead-79de-4129-b2ad-064fca2c8f38 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:01.036193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27eead-79de-4129-b2ad-064fca2c8f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:01.036504 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27eead-79de-4129-b2ad-064fca2c8f38 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:01.037145 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27eead-79de-4129-b2ad-064fca2c8f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:01.039057 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27eead-79de-4129-b2ad-064fca2c8f38 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:01.061194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1122/2237] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:00 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:01.066043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32435bc-67f1-4912-8370-3bf7d8893fde admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:01.077794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32435bc-67f1-4912-8370-3bf7d8893fde admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:01.082755 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13e21b-5bec-4240-9252-b543921db0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:01.083396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13e21b-5bec-4240-9252-b543921db0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:01.083677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13e21b-5bec-4240-9252-b543921db0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:01.083948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13e21b-5bec-4240-9252-b543921db0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:01.084611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13e21b-5bec-4240-9252-b543921db0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:01.086206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13e21b-5bec-4240-9252-b543921db0a7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:01.095927 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1116/2238] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:01.099851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27eead-79de-4129-b2ad-064fca2c8f38 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:01.110648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27eead-79de-4129-b2ad-064fca2c8f38 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:01.115196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ecef9-9d05-4ecc-8f3c-bf17b748e1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:01.115781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ecef9-9d05-4ecc-8f3c-bf17b748e1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:01.116058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ecef9-9d05-4ecc-8f3c-bf17b748e1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:01.116327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ecef9-9d05-4ecc-8f3c-bf17b748e1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:01.116992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ecef9-9d05-4ecc-8f3c-bf17b748e1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:01.118565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ecef9-9d05-4ecc-8f3c-bf17b748e1e6 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:01.129843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1123/2239] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:01.133922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13e21b-5bec-4240-9252-b543921db0a7 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:01.143562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13e21b-5bec-4240-9252-b543921db0a7 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:01.148114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193c13a-3fc5-4500-a250-527384ef7240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:01.148787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193c13a-3fc5-4500-a250-527384ef7240 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:01.149064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193c13a-3fc5-4500-a250-527384ef7240 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:01.149335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193c13a-3fc5-4500-a250-527384ef7240 admin admin] PATH_INFO: `/v3/projects/6f5a94c94a754e5e8f04b5dfd44740af/users/a8371cf44d0143b0a5a3c2f7210fa14e/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:01.158124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:01.166463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193c13a-3fc5-4500-a250-527384ef7240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5a94c94a754e5e8f04b5dfd44740af, user_id=a8371cf44d0143b0a5a3c2f7210fa14e, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:01.169531 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193c13a-3fc5-4500-a250-527384ef7240 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:01.182919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1117/2240] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:01 2022] PUT /identity/v3/projects/6f5a94c94a754e5e8f04b5dfd44740af/users/a8371cf44d0143b0a5a3c2f7210fa14e/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:01.193629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f752-123b-47bc-8f9d-1ec6b7a3308d None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:01.193991 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f752-123b-47bc-8f9d-1ec6b7a3308d None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:01.194292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f752-123b-47bc-8f9d-1ec6b7a3308d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:01.226084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-47f7f752-123b-47bc-8f9d-1ec6b7a3308d None None] MFA Rules not processed for user `a8371cf44d0143b0a5a3c2f7210fa14e`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:01.227530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-47f7f752-123b-47bc-8f9d-1ec6b7a3308d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:01.299914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1124/2241] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:01.311288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193c13a-3fc5-4500-a250-527384ef7240 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:01.313572 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4193c13a-3fc5-4500-a250-527384ef7240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:01.376080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193c13a-3fc5-4500-a250-527384ef7240 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:01.379921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c190-6117-421f-b656-79c1c75497f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:01.380506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c190-6117-421f-b656-79c1c75497f1 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:01.380790 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c190-6117-421f-b656-79c1c75497f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:01.381064 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c190-6117-421f-b656-79c1c75497f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:01.389649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c190-6117-421f-b656-79c1c75497f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:01.391149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c190-6117-421f-b656-79c1c75497f1 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:01.400183 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1118/2242] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:01.424773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7f752-123b-47bc-8f9d-1ec6b7a3308d None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:01.468857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:01.490521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7f752-123b-47bc-8f9d-1ec6b7a3308d None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:01.495196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:01.495828 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:01.496109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:01.496521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] PATH_INFO: `/v3/users/a8371cf44d0143b0a5a3c2f7210fa14e` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:01.497669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8371cf44d0143b0a5a3c2f7210fa14e)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:01.500085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:01.549725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8371cf44d0143b0a5a3c2f7210fa14e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:01.556391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8371cf44d0143b0a5a3c2f7210fa14e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:01.557065 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8371cf44d0143b0a5a3c2f7210fa14e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:01.557687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8371cf44d0143b0a5a3c2f7210fa14e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:01.565347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8371cf44d0143b0a5a3c2f7210fa14e'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:01.584372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1125/2243] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:01 2022] DELETE /identity/v3/users/a8371cf44d0143b0a5a3c2f7210fa14e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:01.588975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f2a59-3aa8-4fb6-a669-d79dd3e66434 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:01.589279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f2a59-3aa8-4fb6-a669-d79dd3e66434 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:01.589570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f2a59-3aa8-4fb6-a669-d79dd3e66434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:01.609755 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5a6f2a59-3aa8-4fb6-a669-d79dd3e66434 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:01.610932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5a6f2a59-3aa8-4fb6-a669-d79dd3e66434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:01.673609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1119/2244] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:01.683365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:01.684825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:01.751548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85945100-26b6-41b6-9ba9-a7eb621e042f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:01.755501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a951d9-e0fb-4fd5-a992-3680514f06f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:01.756139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a951d9-e0fb-4fd5-a992-3680514f06f9 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:01.756468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a951d9-e0fb-4fd5-a992-3680514f06f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:01.756748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a951d9-e0fb-4fd5-a992-3680514f06f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:01.766808 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a951d9-e0fb-4fd5-a992-3680514f06f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:01.768438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a951d9-e0fb-4fd5-a992-3680514f06f9 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:01.778275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1126/2245] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:02.057654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f2a59-3aa8-4fb6-a669-d79dd3e66434 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:02.059340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5a6f2a59-3aa8-4fb6-a669-d79dd3e66434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:02.097656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f2a59-3aa8-4fb6-a669-d79dd3e66434 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:02.104598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:02.105242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.105527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.105806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] PATH_INFO: `/v3/projects/6f5a94c94a754e5e8f04b5dfd44740af` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.106934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5a94c94a754e5e8f04b5dfd44740af)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:02.108921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:02.162042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5a94c94a754e5e8f04b5dfd44740af', 'request_id': 'req-34e7610d-e256-46cb-a689-8ef63861390a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:02.169505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5a94c94a754e5e8f04b5dfd44740af', 'request_id': 'req-34e7610d-e256-46cb-a689-8ef63861390a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:02.177994 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5a94c94a754e5e8f04b5dfd44740af', 'request_id': 'req-34e7610d-e256-46cb-a689-8ef63861390a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:02.190792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] The token cache is being invalidate because project 6f5a94c94a754e5e8f04b5dfd44740af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:02.191408 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:02.191836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-34e7610d-e256-46cb-a689-8ef63861390a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:02.193482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1120/2246] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:02 2022] DELETE /identity/v3/projects/6f5a94c94a754e5e8f04b5dfd44740af => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:02.198488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dad6a3-86d0-4f60-969d-5fbe841e4514 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.198780 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dad6a3-86d0-4f60-969d-5fbe841e4514 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.199059 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dad6a3-86d0-4f60-969d-5fbe841e4514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.218954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-91dad6a3-86d0-4f60-969d-5fbe841e4514 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:02.220095 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-91dad6a3-86d0-4f60-969d-5fbe841e4514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:02.282432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1127/2247] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:02.290404 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5c05-ead2-48dd-92e3-1b7fa84833d7 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.290695 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5c05-ead2-48dd-92e3-1b7fa84833d7 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.290981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5c05-ead2-48dd-92e3-1b7fa84833d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.312955 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-68ca5c05-ead2-48dd-92e3-1b7fa84833d7 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:02.314273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-68ca5c05-ead2-48dd-92e3-1b7fa84833d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:02.322496 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1121/2248] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:48:02.326875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dad6a3-86d0-4f60-969d-5fbe841e4514 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:02.332707 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:02.339944 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dad6a3-86d0-4f60-969d-5fbe841e4514 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:02.357173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d03ce-b042-4cf4-8335-22b62c1b97e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:02.357855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d03ce-b042-4cf4-8335-22b62c1b97e2 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.358137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d03ce-b042-4cf4-8335-22b62c1b97e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.358408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d03ce-b042-4cf4-8335-22b62c1b97e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.359568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d03ce-b042-4cf4-8335-22b62c1b97e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:02.359873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d03ce-b042-4cf4-8335-22b62c1b97e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:02.361571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d03ce-b042-4cf4-8335-22b62c1b97e2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:02.375369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1128/2249] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:48:02.379695 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca5c05-ead2-48dd-92e3-1b7fa84833d7 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:02.389274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca5c05-ead2-48dd-92e3-1b7fa84833d7 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:02.394081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3caf5a-83ce-4e48-8f5f-11fe32ba890e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:02.394700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3caf5a-83ce-4e48-8f5f-11fe32ba890e admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.394976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3caf5a-83ce-4e48-8f5f-11fe32ba890e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.395249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3caf5a-83ce-4e48-8f5f-11fe32ba890e admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.395897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3caf5a-83ce-4e48-8f5f-11fe32ba890e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:02.397857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3caf5a-83ce-4e48-8f5f-11fe32ba890e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:02.402654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:02.429969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1122/2250] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:02 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:02.434488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d03ce-b042-4cf4-8335-22b62c1b97e2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:02.444516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d03ce-b042-4cf4-8335-22b62c1b97e2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:02.499521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba804df-99d1-4875-8530-09cd013a6825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:02.500158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba804df-99d1-4875-8530-09cd013a6825 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.500471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba804df-99d1-4875-8530-09cd013a6825 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.500747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba804df-99d1-4875-8530-09cd013a6825 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.501420 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba804df-99d1-4875-8530-09cd013a6825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:02.503337 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba804df-99d1-4875-8530-09cd013a6825 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:02.526349 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1129/2251] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:02 2022] POST /identity/v3/users => generated 280 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:02.530698 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3caf5a-83ce-4e48-8f5f-11fe32ba890e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:02.542363 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3caf5a-83ce-4e48-8f5f-11fe32ba890e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:02.547258 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70b65e-842c-4a06-999f-8b47edc5b64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:02.547855 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70b65e-842c-4a06-999f-8b47edc5b64f admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.548134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70b65e-842c-4a06-999f-8b47edc5b64f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.548437 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70b65e-842c-4a06-999f-8b47edc5b64f admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.549154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70b65e-842c-4a06-999f-8b47edc5b64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:02.550804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70b65e-842c-4a06-999f-8b47edc5b64f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:02.564391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1123/2252] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:02.565462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba804df-99d1-4875-8530-09cd013a6825 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:02.575890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba804df-99d1-4875-8530-09cd013a6825 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:02.582026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d10308-eec4-475b-9bcb-0681dc2cb9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:02.582194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d10308-eec4-475b-9bcb-0681dc2cb9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.582448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d10308-eec4-475b-9bcb-0681dc2cb9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.582733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d10308-eec4-475b-9bcb-0681dc2cb9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.583383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d10308-eec4-475b-9bcb-0681dc2cb9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:02.585045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d10308-eec4-475b-9bcb-0681dc2cb9c7 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:02.597009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1130/2253] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:02.601562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70b65e-842c-4a06-999f-8b47edc5b64f admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:02.612289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70b65e-842c-4a06-999f-8b47edc5b64f admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:02.617133 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b68e8-c886-4a53-918d-bef55b970054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:02.617763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b68e8-c886-4a53-918d-bef55b970054 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.618041 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b68e8-c886-4a53-918d-bef55b970054 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.618371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b68e8-c886-4a53-918d-bef55b970054 admin admin] PATH_INFO: `/v3/projects/ca922823d2f347498592bfe10a3b4c2b/users/ad7ddd477fc44aef9478c5dad6443931/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.635146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b68e8-c886-4a53-918d-bef55b970054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca922823d2f347498592bfe10a3b4c2b, user_id=ad7ddd477fc44aef9478c5dad6443931, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:02.641284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b68e8-c886-4a53-918d-bef55b970054 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:02.654603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1124/2254] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:02 2022] PUT /identity/v3/projects/ca922823d2f347498592bfe10a3b4c2b/users/ad7ddd477fc44aef9478c5dad6443931/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:02.666842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd98e73-5fc1-4388-9e0d-8397e0225a2b None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.667341 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd98e73-5fc1-4388-9e0d-8397e0225a2b None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.667840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd98e73-5fc1-4388-9e0d-8397e0225a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.698094 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1bd98e73-5fc1-4388-9e0d-8397e0225a2b None None] MFA Rules not processed for user `ad7ddd477fc44aef9478c5dad6443931`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:02.699309 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1bd98e73-5fc1-4388-9e0d-8397e0225a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:02.766511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1131/2255] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:02.804625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b68e8-c886-4a53-918d-bef55b970054 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:02.806447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1c6b68e8-c886-4a53-918d-bef55b970054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:02.866242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b68e8-c886-4a53-918d-bef55b970054 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:02.870131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9271a85-4f31-454d-a373-fb00ba32556f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:02.870731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9271a85-4f31-454d-a373-fb00ba32556f service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.871577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9271a85-4f31-454d-a373-fb00ba32556f service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.871864 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9271a85-4f31-454d-a373-fb00ba32556f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.880059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9271a85-4f31-454d-a373-fb00ba32556f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:02.881524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9271a85-4f31-454d-a373-fb00ba32556f service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:02.890311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1125/2256] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:02.906323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd98e73-5fc1-4388-9e0d-8397e0225a2b None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:02.963160 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:02.967476 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd98e73-5fc1-4388-9e0d-8397e0225a2b None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:02.973456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:02.974080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:02.974357 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:02.974631 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] PATH_INFO: `/v3/users/ad7ddd477fc44aef9478c5dad6443931` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:02.975743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7ddd477fc44aef9478c5dad6443931)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:02.977690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:03.023158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7ddd477fc44aef9478c5dad6443931'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.029872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7ddd477fc44aef9478c5dad6443931'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.030517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7ddd477fc44aef9478c5dad6443931'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.031322 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7ddd477fc44aef9478c5dad6443931'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.038291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7ddd477fc44aef9478c5dad6443931'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.055731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1132/2257] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:02 2022] DELETE /identity/v3/users/ad7ddd477fc44aef9478c5dad6443931 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:03.060208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42c28-13a9-499b-a3e6-28ae48b9ad12 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:03.060541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42c28-13a9-499b-a3e6-28ae48b9ad12 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:03.060822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42c28-13a9-499b-a3e6-28ae48b9ad12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:03.080931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6ea42c28-13a9-499b-a3e6-28ae48b9ad12 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:03.082134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6ea42c28-13a9-499b-a3e6-28ae48b9ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:03.139577 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1126/2258] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:03.149722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:03.151120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:03.207616 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6d5cd-94f3-4f27-99d5-8ae74d86f07b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:03.211285 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777aa61-a174-4ccb-9b55-9e23c525bd3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:03.211884 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777aa61-a174-4ccb-9b55-9e23c525bd3c service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:03.212161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777aa61-a174-4ccb-9b55-9e23c525bd3c service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:03.212459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777aa61-a174-4ccb-9b55-9e23c525bd3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:03.221368 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777aa61-a174-4ccb-9b55-9e23c525bd3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:03.222783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777aa61-a174-4ccb-9b55-9e23c525bd3c service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:03.232504 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1133/2259] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:03.553662 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42c28-13a9-499b-a3e6-28ae48b9ad12 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:03.555410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6ea42c28-13a9-499b-a3e6-28ae48b9ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:03.573580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42c28-13a9-499b-a3e6-28ae48b9ad12 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:03.580018 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:03.580883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:03.581166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:03.581441 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] PATH_INFO: `/v3/projects/ca922823d2f347498592bfe10a3b4c2b` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:03.582613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca922823d2f347498592bfe10a3b4c2b)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:03.584678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:03.635545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:03.636952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca922823d2f347498592bfe10a3b4c2b', 'request_id': 'req-6039392d-900b-4c5d-98f4-9bbfbbbf9685'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.644397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca922823d2f347498592bfe10a3b4c2b', 'request_id': 'req-6039392d-900b-4c5d-98f4-9bbfbbbf9685'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.652207 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca922823d2f347498592bfe10a3b4c2b', 'request_id': 'req-6039392d-900b-4c5d-98f4-9bbfbbbf9685'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.664417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] The token cache is being invalidate because project ca922823d2f347498592bfe10a3b4c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:03.664798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.665545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6039392d-900b-4c5d-98f4-9bbfbbbf9685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.667222 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1127/2260] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:03 2022] DELETE /identity/v3/projects/ca922823d2f347498592bfe10a3b4c2b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:03.672461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b591c-1437-4240-8381-fcc7c41e35b6 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:03.672951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b591c-1437-4240-8381-fcc7c41e35b6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:03.673067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b591c-1437-4240-8381-fcc7c41e35b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:03.694237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4e2b591c-1437-4240-8381-fcc7c41e35b6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:03.695324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4e2b591c-1437-4240-8381-fcc7c41e35b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:03.758820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1134/2261] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:03.767162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37244e9a-10ce-4099-9974-4fff26c01c50 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:03.767454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37244e9a-10ce-4099-9974-4fff26c01c50 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:03.767740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37244e9a-10ce-4099-9974-4fff26c01c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:03.772914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b591c-1437-4240-8381-fcc7c41e35b6 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:03.774292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4e2b591c-1437-4240-8381-fcc7c41e35b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:03.788060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-37244e9a-10ce-4099-9974-4fff26c01c50 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:03.788531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b591c-1437-4240-8381-fcc7c41e35b6 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:03.789288 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-37244e9a-10ce-4099-9974-4fff26c01c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:03.793083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:03.793807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:03.794089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:03.794358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] PATH_INFO: `/v3/users/a0d8e696ac234d2dafbd82d07a373d9c` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:03.795456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d8e696ac234d2dafbd82d07a373d9c)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:03.797907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:03.801438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1128/2262] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:03.802279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37244e9a-10ce-4099-9974-4fff26c01c50 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:03.811267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37244e9a-10ce-4099-9974-4fff26c01c50 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:03.811800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:03.816481 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdf0df-621a-4dcd-805b-6de071d22414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:03.817087 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdf0df-621a-4dcd-805b-6de071d22414 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:03.817371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdf0df-621a-4dcd-805b-6de071d22414 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:03.817649 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdf0df-621a-4dcd-805b-6de071d22414 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:03.818328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdf0df-621a-4dcd-805b-6de071d22414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:03.818635 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdf0df-621a-4dcd-805b-6de071d22414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:03.820250 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdf0df-621a-4dcd-805b-6de071d22414 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:03.834820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1129/2263] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:48:03.839841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdf0df-621a-4dcd-805b-6de071d22414 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:03.851981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdf0df-621a-4dcd-805b-6de071d22414 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:03.855516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d8e696ac234d2dafbd82d07a373d9c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.857309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee66ecf-c292-4d75-8288-6d114a9257f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:03.858071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee66ecf-c292-4d75-8288-6d114a9257f1 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:03.858342 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee66ecf-c292-4d75-8288-6d114a9257f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:03.858681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee66ecf-c292-4d75-8288-6d114a9257f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:03.859416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee66ecf-c292-4d75-8288-6d114a9257f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:03.861396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee66ecf-c292-4d75-8288-6d114a9257f1 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:03.864277 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d8e696ac234d2dafbd82d07a373d9c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.865066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d8e696ac234d2dafbd82d07a373d9c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.865764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d8e696ac234d2dafbd82d07a373d9c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.876210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-46dc890d-2145-4d1d-8777-251170821da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d8e696ac234d2dafbd82d07a373d9c'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:03.899305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1130/2264] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:03 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:03.903488 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee66ecf-c292-4d75-8288-6d114a9257f1 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:03.907871 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1135/2265] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:03 2022] DELETE /identity/v3/users/a0d8e696ac234d2dafbd82d07a373d9c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:03.908723 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8ee66ecf-c292-4d75-8288-6d114a9257f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:03.909414 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103afd4-f734-431e-aed0-47b9a36018cb None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:03.909770 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103afd4-f734-431e-aed0-47b9a36018cb None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:03.910204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103afd4-f734-431e-aed0-47b9a36018cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:03.932868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e103afd4-f734-431e-aed0-47b9a36018cb None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:03.934110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e103afd4-f734-431e-aed0-47b9a36018cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:03.991065 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee66ecf-c292-4d75-8288-6d114a9257f1 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:04.035464 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1136/2266] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:04.045500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103afd4-f734-431e-aed0-47b9a36018cb None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:04.046997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e103afd4-f734-431e-aed0-47b9a36018cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:04.071679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625663dc-d7fb-4cdd-9aee-4b093a896501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:04.072540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625663dc-d7fb-4cdd-9aee-4b093a896501 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:04.072821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625663dc-d7fb-4cdd-9aee-4b093a896501 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:04.073154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625663dc-d7fb-4cdd-9aee-4b093a896501 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:04.074260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625663dc-d7fb-4cdd-9aee-4b093a896501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:04.077068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625663dc-d7fb-4cdd-9aee-4b093a896501 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:04.104703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1131/2267] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:03 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:04.105777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625663dc-d7fb-4cdd-9aee-4b093a896501 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:04.116457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103afd4-f734-431e-aed0-47b9a36018cb None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:04.117820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625663dc-d7fb-4cdd-9aee-4b093a896501 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:04.121470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978aff6-a852-4607-b9bf-e0df7e1295b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:04.122652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978aff6-a852-4607-b9bf-e0df7e1295b6 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:04.123343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978aff6-a852-4607-b9bf-e0df7e1295b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:04.123726 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978aff6-a852-4607-b9bf-e0df7e1295b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:04.123824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f668a-2c75-4704-b138-a469ad819349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:04.124958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f668a-2c75-4704-b138-a469ad819349 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:04.125249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f668a-2c75-4704-b138-a469ad819349 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:04.125760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f668a-2c75-4704-b138-a469ad819349 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:04.126396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f668a-2c75-4704-b138-a469ad819349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:04.128004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f668a-2c75-4704-b138-a469ad819349 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:04.139003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978aff6-a852-4607-b9bf-e0df7e1295b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:04.139991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1132/2268] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:04.141005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978aff6-a852-4607-b9bf-e0df7e1295b6 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:04.144935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f668a-2c75-4704-b138-a469ad819349 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:04.152606 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1137/2269] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:04.160467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f668a-2c75-4704-b138-a469ad819349 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:04.166733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d6ccb-acaf-47f2-8de8-b1b3fdf81a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:04.169048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d6ccb-acaf-47f2-8de8-b1b3fdf81a70 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:04.169454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d6ccb-acaf-47f2-8de8-b1b3fdf81a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:04.169805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d6ccb-acaf-47f2-8de8-b1b3fdf81a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:04.171803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d6ccb-acaf-47f2-8de8-b1b3fdf81a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:04.174499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d6ccb-acaf-47f2-8de8-b1b3fdf81a70 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:04.188641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1133/2270] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:04.192786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978aff6-a852-4607-b9bf-e0df7e1295b6 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:04.204809 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978aff6-a852-4607-b9bf-e0df7e1295b6 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:04.209675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5c76f-d554-40d5-8229-8b5ab712cf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:04.210370 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5c76f-d554-40d5-8229-8b5ab712cf3e admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:04.210652 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5c76f-d554-40d5-8229-8b5ab712cf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:04.211040 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5c76f-d554-40d5-8229-8b5ab712cf3e admin admin] PATH_INFO: `/v3/projects/a4de79529fb44a05a7842b6940c4e342/users/b64df7e3be404afcb30726783fa936b8/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:04.234946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5c76f-d554-40d5-8229-8b5ab712cf3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4de79529fb44a05a7842b6940c4e342, user_id=b64df7e3be404afcb30726783fa936b8, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:04.243210 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5c76f-d554-40d5-8229-8b5ab712cf3e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:04.262378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1138/2271] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:04 2022] PUT /identity/v3/projects/a4de79529fb44a05a7842b6940c4e342/users/b64df7e3be404afcb30726783fa936b8/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:04.272938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0526faf-f6f4-4ad8-86d5-a1fb143a6e1b None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:04.273225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0526faf-f6f4-4ad8-86d5-a1fb143a6e1b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:04.273510 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0526faf-f6f4-4ad8-86d5-a1fb143a6e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:04.297210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:04.309088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d0526faf-f6f4-4ad8-86d5-a1fb143a6e1b None None] MFA Rules not processed for user `b64df7e3be404afcb30726783fa936b8`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:04.310852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d0526faf-f6f4-4ad8-86d5-a1fb143a6e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:04.413578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1134/2272] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:04.425803 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5c76f-d554-40d5-8229-8b5ab712cf3e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:04.427600 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a1a5c76f-d554-40d5-8229-8b5ab712cf3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:04.442114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:04.487116 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0526faf-f6f4-4ad8-86d5-a1fb143a6e1b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:04.489029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d0526faf-f6f4-4ad8-86d5-a1fb143a6e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:04.491391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5c76f-d554-40d5-8229-8b5ab712cf3e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:04.495879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b936b-5282-4203-868b-6feae4bf635c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:04.496638 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b936b-5282-4203-868b-6feae4bf635c service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:04.496922 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b936b-5282-4203-868b-6feae4bf635c service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:04.497198 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b936b-5282-4203-868b-6feae4bf635c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:04.521347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b936b-5282-4203-868b-6feae4bf635c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:04.522810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b936b-5282-4203-868b-6feae4bf635c service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:04.533887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1139/2273] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:04.564735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0526faf-f6f4-4ad8-86d5-a1fb143a6e1b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:04.570002 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:04.571259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:04.571893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:04.572401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] PATH_INFO: `/v3/projects/76c10ed1f7274fe8b3df08c0150dcae3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:04.574031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c10ed1f7274fe8b3df08c0150dcae3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:04.576369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:04.636621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c10ed1f7274fe8b3df08c0150dcae3', 'request_id': 'req-176af389-14de-4a0c-aae4-c126ddc4d154'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.645157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c10ed1f7274fe8b3df08c0150dcae3', 'request_id': 'req-176af389-14de-4a0c-aae4-c126ddc4d154'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.653980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c10ed1f7274fe8b3df08c0150dcae3', 'request_id': 'req-176af389-14de-4a0c-aae4-c126ddc4d154'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.668876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] The token cache is being invalidate because project 76c10ed1f7274fe8b3df08c0150dcae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:04.669200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.669889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.671684 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1135/2274] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:04 2022] DELETE /identity/v3/projects/76c10ed1f7274fe8b3df08c0150dcae3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:04.676308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b936b-5282-4203-868b-6feae4bf635c service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:04.691826 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b936b-5282-4203-868b-6feae4bf635c service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:04.767450 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:04.768070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:04.768762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:04.768762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] PATH_INFO: `/v3/users/2980ef1f198a44d3acc7555193185594` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:04.769912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2980ef1f198a44d3acc7555193185594)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:04.772046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:04.832745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2980ef1f198a44d3acc7555193185594'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.840670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2980ef1f198a44d3acc7555193185594'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.841514 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2980ef1f198a44d3acc7555193185594'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.842129 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2980ef1f198a44d3acc7555193185594'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.850437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a797f16a-450e-426d-a9aa-2ae5532ab6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2980ef1f198a44d3acc7555193185594'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:04.877298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1140/2275] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:04 2022] DELETE /identity/v3/users/2980ef1f198a44d3acc7555193185594 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:05.148862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:05.150459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:05.163656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176af389-14de-4a0c-aae4-c126ddc4d154 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:05.194127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:05.226220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:05.226848 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.227132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.227409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] PATH_INFO: `/v3/projects/6319f793b1404a74a4071c6328c9ba77` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.228804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6319f793b1404a74a4071c6328c9ba77)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:05.230713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:05.277223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6319f793b1404a74a4071c6328c9ba77', 'request_id': 'req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:05.283101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6319f793b1404a74a4071c6328c9ba77', 'request_id': 'req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:05.290365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6319f793b1404a74a4071c6328c9ba77', 'request_id': 'req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:05.303642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] The token cache is being invalidate because project 6319f793b1404a74a4071c6328c9ba77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:05.303934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:05.304615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f6ed4cb0-afa7-41ae-ba61-c056f7b6406e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:05.306155 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1136/2276] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:05 2022] DELETE /identity/v3/projects/6319f793b1404a74a4071c6328c9ba77 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:05.311016 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf64604-6d62-4206-91aa-f3448b607d0e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.311302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf64604-6d62-4206-91aa-f3448b607d0e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.311579 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf64604-6d62-4206-91aa-f3448b607d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.331606 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-baf64604-6d62-4206-91aa-f3448b607d0e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:05.332825 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-baf64604-6d62-4206-91aa-f3448b607d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:05.363008 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:05.395457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1141/2277] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:05.403187 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145389e5-31c8-4ebd-b2bb-13dacf3153ea None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.403540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145389e5-31c8-4ebd-b2bb-13dacf3153ea None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.403879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145389e5-31c8-4ebd-b2bb-13dacf3153ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.425598 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-145389e5-31c8-4ebd-b2bb-13dacf3153ea None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:05.426986 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-145389e5-31c8-4ebd-b2bb-13dacf3153ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:05.434991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1137/2278] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:05.439372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf64604-6d62-4206-91aa-f3448b607d0e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:05.449712 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf64604-6d62-4206-91aa-f3448b607d0e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:05.455035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950df7f-08e1-4c74-b9d7-1a1d17ff2eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:05.455664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950df7f-08e1-4c74-b9d7-1a1d17ff2eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.455940 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950df7f-08e1-4c74-b9d7-1a1d17ff2eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.456208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950df7f-08e1-4c74-b9d7-1a1d17ff2eb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.456892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950df7f-08e1-4c74-b9d7-1a1d17ff2eb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:05.457273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950df7f-08e1-4c74-b9d7-1a1d17ff2eb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:05.459356 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950df7f-08e1-4c74-b9d7-1a1d17ff2eb8 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:05.474645 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1142/2279] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:48:05.475717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145389e5-31c8-4ebd-b2bb-13dacf3153ea None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:05.485997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145389e5-31c8-4ebd-b2bb-13dacf3153ea None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:05.490631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3387cd-de73-4b5f-92fc-459f0302331c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:05.491221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3387cd-de73-4b5f-92fc-459f0302331c admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.491499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3387cd-de73-4b5f-92fc-459f0302331c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.491772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3387cd-de73-4b5f-92fc-459f0302331c admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.492443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3387cd-de73-4b5f-92fc-459f0302331c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:05.494316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3387cd-de73-4b5f-92fc-459f0302331c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:05.523521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1138/2280] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:05 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:05.527851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950df7f-08e1-4c74-b9d7-1a1d17ff2eb8 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:05.538823 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950df7f-08e1-4c74-b9d7-1a1d17ff2eb8 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:05.602144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c372457-b927-4e76-bc8e-90f7deaead2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:05.602750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c372457-b927-4e76-bc8e-90f7deaead2c admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.603031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c372457-b927-4e76-bc8e-90f7deaead2c admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.603307 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c372457-b927-4e76-bc8e-90f7deaead2c admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.604229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c372457-b927-4e76-bc8e-90f7deaead2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:05.606169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c372457-b927-4e76-bc8e-90f7deaead2c admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:05.627563 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1143/2281] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:05 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:05.631697 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3387cd-de73-4b5f-92fc-459f0302331c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:05.642829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3387cd-de73-4b5f-92fc-459f0302331c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:05.648216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cae72a-b06c-4a0e-8e20-4d85ff0751aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:05.648851 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cae72a-b06c-4a0e-8e20-4d85ff0751aa admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.649128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cae72a-b06c-4a0e-8e20-4d85ff0751aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.649397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cae72a-b06c-4a0e-8e20-4d85ff0751aa admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.650058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cae72a-b06c-4a0e-8e20-4d85ff0751aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:05.651640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cae72a-b06c-4a0e-8e20-4d85ff0751aa admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:05.665387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1139/2282] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:05.670278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c372457-b927-4e76-bc8e-90f7deaead2c admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:05.681045 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c372457-b927-4e76-bc8e-90f7deaead2c admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:05.686806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f522fb-2676-41f6-b0c8-69ca66b37a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:05.688610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f522fb-2676-41f6-b0c8-69ca66b37a99 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.689677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f522fb-2676-41f6-b0c8-69ca66b37a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.690032 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f522fb-2676-41f6-b0c8-69ca66b37a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.690925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f522fb-2676-41f6-b0c8-69ca66b37a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:05.692952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f522fb-2676-41f6-b0c8-69ca66b37a99 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:05.711623 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1144/2283] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:05.712600 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cae72a-b06c-4a0e-8e20-4d85ff0751aa admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:05.722850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cae72a-b06c-4a0e-8e20-4d85ff0751aa admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:05.729588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc7356-9d76-4de9-9ed9-08004fe4b097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:05.730347 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc7356-9d76-4de9-9ed9-08004fe4b097 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.730694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc7356-9d76-4de9-9ed9-08004fe4b097 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.731032 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc7356-9d76-4de9-9ed9-08004fe4b097 admin admin] PATH_INFO: `/v3/projects/1e6f184a37f64d58a055953fad43ce9a/users/38375e7dc8504f898c25c1fcdf14f89e/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.747608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc7356-9d76-4de9-9ed9-08004fe4b097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6f184a37f64d58a055953fad43ce9a, user_id=38375e7dc8504f898c25c1fcdf14f89e, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:05.751487 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc7356-9d76-4de9-9ed9-08004fe4b097 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:05.766323 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1140/2284] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:05 2022] PUT /identity/v3/projects/1e6f184a37f64d58a055953fad43ce9a/users/38375e7dc8504f898c25c1fcdf14f89e/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:05.777607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cad636-60c2-46bf-9890-859dd79e361f None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:05.777951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cad636-60c2-46bf-9890-859dd79e361f None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:05.778170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cad636-60c2-46bf-9890-859dd79e361f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:05.824914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-e4cad636-60c2-46bf-9890-859dd79e361f None None] MFA Rules not processed for user `38375e7dc8504f898c25c1fcdf14f89e`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:05.830831 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e4cad636-60c2-46bf-9890-859dd79e361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:05.907378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1145/2285] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:05.959980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc7356-9d76-4de9-9ed9-08004fe4b097 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:05.961923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e7fc7356-9d76-4de9-9ed9-08004fe4b097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:06.023808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc7356-9d76-4de9-9ed9-08004fe4b097 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:06.028866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac50451-a63f-40fb-87c8-433a8891118e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:06.029418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac50451-a63f-40fb-87c8-433a8891118e service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:06.029707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac50451-a63f-40fb-87c8-433a8891118e service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:06.029984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac50451-a63f-40fb-87c8-433a8891118e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:06.039714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac50451-a63f-40fb-87c8-433a8891118e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:06.041070 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac50451-a63f-40fb-87c8-433a8891118e service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:06.044787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:06.051654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1141/2286] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:07.371568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cad636-60c2-46bf-9890-859dd79e361f None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:07.373090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e4cad636-60c2-46bf-9890-859dd79e361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:07.408500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:07.458488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cad636-60c2-46bf-9890-859dd79e361f None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:07.463852 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:07.464474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:07.464758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:07.465034 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] PATH_INFO: `/v3/users/b64df7e3be404afcb30726783fa936b8` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:07.466169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64df7e3be404afcb30726783fa936b8)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:07.468128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:07.526887 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64df7e3be404afcb30726783fa936b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:07.534345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64df7e3be404afcb30726783fa936b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:07.535248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64df7e3be404afcb30726783fa936b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:07.535864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64df7e3be404afcb30726783fa936b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:07.543748 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64df7e3be404afcb30726783fa936b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:07.564381 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1146/2287] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:07 2022] DELETE /identity/v3/users/b64df7e3be404afcb30726783fa936b8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:07.569076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397f5f4-eb76-4852-935e-ee0b9ceeb223 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:07.569617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397f5f4-eb76-4852-935e-ee0b9ceeb223 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:07.569705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397f5f4-eb76-4852-935e-ee0b9ceeb223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:07.592674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-1397f5f4-eb76-4852-935e-ee0b9ceeb223 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:07.594441 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1397f5f4-eb76-4852-935e-ee0b9ceeb223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:07.664390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1142/2288] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:07.672245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:07.673936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:07.733243 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cb47f-7527-4375-97f9-a17bb6d18373 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:07.737174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171a948-a3ee-45b5-a496-6aabde6ceca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:07.737769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171a948-a3ee-45b5-a496-6aabde6ceca1 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:07.738072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171a948-a3ee-45b5-a496-6aabde6ceca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:07.738343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171a948-a3ee-45b5-a496-6aabde6ceca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:07.747312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171a948-a3ee-45b5-a496-6aabde6ceca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:07.748771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171a948-a3ee-45b5-a496-6aabde6ceca1 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:07.758924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1147/2289] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:08.027758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397f5f4-eb76-4852-935e-ee0b9ceeb223 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:08.029278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1397f5f4-eb76-4852-935e-ee0b9ceeb223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:08.047919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397f5f4-eb76-4852-935e-ee0b9ceeb223 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:08.053998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:08.054658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.055020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.055252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] PATH_INFO: `/v3/projects/a4de79529fb44a05a7842b6940c4e342` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.056963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4de79529fb44a05a7842b6940c4e342)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:08.059025 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:08.113263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4de79529fb44a05a7842b6940c4e342', 'request_id': 'req-8686e6fb-9139-4cc9-8655-4580312af3e4'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:08.118785 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4de79529fb44a05a7842b6940c4e342', 'request_id': 'req-8686e6fb-9139-4cc9-8655-4580312af3e4'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:08.126922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4de79529fb44a05a7842b6940c4e342', 'request_id': 'req-8686e6fb-9139-4cc9-8655-4580312af3e4'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:08.138981 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] The token cache is being invalidate because project a4de79529fb44a05a7842b6940c4e342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:08.139269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:08.140224 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8686e6fb-9139-4cc9-8655-4580312af3e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:08.142305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1143/2290] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:08 2022] DELETE /identity/v3/projects/a4de79529fb44a05a7842b6940c4e342 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:08.147530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b92777-8414-4aa4-b547-55e03b43f9c8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.147822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b92777-8414-4aa4-b547-55e03b43f9c8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.148100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b92777-8414-4aa4-b547-55e03b43f9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.167812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-a3b92777-8414-4aa4-b547-55e03b43f9c8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:08.169025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a3b92777-8414-4aa4-b547-55e03b43f9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:08.212985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:08.229027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1148/2291] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:08.237312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84600db2-a1b8-49fc-ae23-30188b5a458e None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.237606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84600db2-a1b8-49fc-ae23-30188b5a458e None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.237897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84600db2-a1b8-49fc-ae23-30188b5a458e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.258671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-84600db2-a1b8-49fc-ae23-30188b5a458e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:08.260013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-84600db2-a1b8-49fc-ae23-30188b5a458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:08.268287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1144/2292] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:08.272424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b92777-8414-4aa4-b547-55e03b43f9c8 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:08.282581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b92777-8414-4aa4-b547-55e03b43f9c8 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:08.287207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8a200-a3b3-4466-a834-af0c1a3b03a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:08.287801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a200-a3b3-4466-a834-af0c1a3b03a2 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.288083 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a200-a3b3-4466-a834-af0c1a3b03a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.288384 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a200-a3b3-4466-a834-af0c1a3b03a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.289031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8a200-a3b3-4466-a834-af0c1a3b03a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:08.289339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8a200-a3b3-4466-a834-af0c1a3b03a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:08.290920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8a200-a3b3-4466-a834-af0c1a3b03a2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:08.302174 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1149/2293] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:48:08.306262 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84600db2-a1b8-49fc-ae23-30188b5a458e None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:08.315328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84600db2-a1b8-49fc-ae23-30188b5a458e None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:08.319894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c146324-67bd-4bda-adc1-4a124e0bb542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:08.320509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c146324-67bd-4bda-adc1-4a124e0bb542 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.320784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c146324-67bd-4bda-adc1-4a124e0bb542 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.321053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c146324-67bd-4bda-adc1-4a124e0bb542 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.321672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c146324-67bd-4bda-adc1-4a124e0bb542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:08.323547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c146324-67bd-4bda-adc1-4a124e0bb542 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:08.352960 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1145/2294] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:08 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:08.357409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8a200-a3b3-4466-a834-af0c1a3b03a2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:08.368085 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8a200-a3b3-4466-a834-af0c1a3b03a2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:08.423722 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e242060-0630-4404-8772-bd5faf789a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:08.424330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e242060-0630-4404-8772-bd5faf789a7e admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.424667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e242060-0630-4404-8772-bd5faf789a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.424938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e242060-0630-4404-8772-bd5faf789a7e admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.425581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e242060-0630-4404-8772-bd5faf789a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:08.427505 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e242060-0630-4404-8772-bd5faf789a7e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:08.450418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1150/2295] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:08 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:08.452952 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c146324-67bd-4bda-adc1-4a124e0bb542 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:08.462973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c146324-67bd-4bda-adc1-4a124e0bb542 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:08.467466 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddea203-6a67-40da-934b-7e29d8ced56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:08.468088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddea203-6a67-40da-934b-7e29d8ced56e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.468415 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddea203-6a67-40da-934b-7e29d8ced56e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.468690 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddea203-6a67-40da-934b-7e29d8ced56e admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.469320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddea203-6a67-40da-934b-7e29d8ced56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:08.470897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddea203-6a67-40da-934b-7e29d8ced56e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:08.481181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1146/2296] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:08.485550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e242060-0630-4404-8772-bd5faf789a7e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:08.495187 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e242060-0630-4404-8772-bd5faf789a7e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:08.500303 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aafdf-0207-461a-b33c-2921f343db21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:08.500924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aafdf-0207-461a-b33c-2921f343db21 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.501202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aafdf-0207-461a-b33c-2921f343db21 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.501473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aafdf-0207-461a-b33c-2921f343db21 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.502106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80aafdf-0207-461a-b33c-2921f343db21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:08.503669 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80aafdf-0207-461a-b33c-2921f343db21 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:08.517491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1151/2297] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:08.518735 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddea203-6a67-40da-934b-7e29d8ced56e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:08.528935 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddea203-6a67-40da-934b-7e29d8ced56e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:08.533985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2f149-1124-45e5-8adb-ad68a6956eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:08.534638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2f149-1124-45e5-8adb-ad68a6956eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.534918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2f149-1124-45e5-8adb-ad68a6956eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.535195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2f149-1124-45e5-8adb-ad68a6956eb9 admin admin] PATH_INFO: `/v3/projects/c4539f07e24f42e18507a1cc54020231/users/296f2991d00c4a17b30298d9a7dc8bf1/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.550239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2f149-1124-45e5-8adb-ad68a6956eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4539f07e24f42e18507a1cc54020231, user_id=296f2991d00c4a17b30298d9a7dc8bf1, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:08.552921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2f149-1124-45e5-8adb-ad68a6956eb9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:08.567086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1147/2298] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:08 2022] PUT /identity/v3/projects/c4539f07e24f42e18507a1cc54020231/users/296f2991d00c4a17b30298d9a7dc8bf1/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:08.577409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e382b-0069-44ab-a2fc-5687ed9d58bb None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.577693 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e382b-0069-44ab-a2fc-5687ed9d58bb None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.577965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e382b-0069-44ab-a2fc-5687ed9d58bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.606313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-007e382b-0069-44ab-a2fc-5687ed9d58bb None None] MFA Rules not processed for user `296f2991d00c4a17b30298d9a7dc8bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:08.607507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-007e382b-0069-44ab-a2fc-5687ed9d58bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:08.670997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1152/2299] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:08.681353 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2f149-1124-45e5-8adb-ad68a6956eb9 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:08.683057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5ce2f149-1124-45e5-8adb-ad68a6956eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:08.709358 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:08.746729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2f149-1124-45e5-8adb-ad68a6956eb9 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:08.750965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee7c8d-ad10-4edc-9cee-01e5edf73a89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:08.751565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee7c8d-ad10-4edc-9cee-01e5edf73a89 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:08.751973 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee7c8d-ad10-4edc-9cee-01e5edf73a89 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:08.752243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee7c8d-ad10-4edc-9cee-01e5edf73a89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:08.760316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee7c8d-ad10-4edc-9cee-01e5edf73a89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:08.761768 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee7c8d-ad10-4edc-9cee-01e5edf73a89 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:08.771027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1148/2300] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:10.798326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e382b-0069-44ab-a2fc-5687ed9d58bb None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:10.799767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-007e382b-0069-44ab-a2fc-5687ed9d58bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:10.811685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e382b-0069-44ab-a2fc-5687ed9d58bb None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:10.815460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:10.816053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:10.816326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:10.816628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:10.818455 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:10.863670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:10.878518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:10.879946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:10.889046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1153/2301] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:18.767435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=18 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:20.884692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=7 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:28.253733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8456a1-7961-4a36-955a-b837c964683b None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:28.254259 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8456a1-7961-4a36-955a-b837c964683b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:28.254498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8456a1-7961-4a36-955a-b837c964683b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:28.277553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-ae8456a1-7961-4a36-955a-b837c964683b None None] MFA Rules not processed for user `ac5921418d40446c80892b04a3eee12d`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:28.278784 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ae8456a1-7961-4a36-955a-b837c964683b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:28.285859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1149/2302] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:48:28.621080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:28.622619 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:28.682383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a3399-5856-41d2-bda4-f2574c55a00c service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:28.686078 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded15ddd-77a6-43aa-af10-0003d80f0f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:28.686766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded15ddd-77a6-43aa-af10-0003d80f0f28 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:28.687122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded15ddd-77a6-43aa-af10-0003d80f0f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:28.687398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded15ddd-77a6-43aa-af10-0003d80f0f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:28.689840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ded15ddd-77a6-43aa-af10-0003d80f0f28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:28.699694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded15ddd-77a6-43aa-af10-0003d80f0f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:28.701170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded15ddd-77a6-43aa-af10-0003d80f0f28 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:28.709621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1154/2303] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:29.979502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8456a1-7961-4a36-955a-b837c964683b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:29.981304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ae8456a1-7961-4a36-955a-b837c964683b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:30.064536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8456a1-7961-4a36-955a-b837c964683b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:30.072177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:30.073047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:30.073578 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:30.074081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] PATH_INFO: `/v3/users/f3194340fe774d72bba97563b73317ec` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:30.075917 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3194340fe774d72bba97563b73317ec)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:30.078167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:30.162693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3194340fe774d72bba97563b73317ec'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.182756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3194340fe774d72bba97563b73317ec'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.183520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3194340fe774d72bba97563b73317ec'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.184132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3194340fe774d72bba97563b73317ec'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.214399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3194340fe774d72bba97563b73317ec'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.245774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1150/2304] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:29 2022] DELETE /identity/v3/users/f3194340fe774d72bba97563b73317ec => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:30.250062 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b53dcb-0b8b-4b41-ae72-7407dec12cf8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:30.250543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b53dcb-0b8b-4b41-ae72-7407dec12cf8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:30.250767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b53dcb-0b8b-4b41-ae72-7407dec12cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:30.272680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-88b53dcb-0b8b-4b41-ae72-7407dec12cf8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:30.273849 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-88b53dcb-0b8b-4b41-ae72-7407dec12cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:30.363647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1155/2305] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:30.377941 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:30.381252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:30.455463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0492f-8d94-4d70-848b-38bccd0ffc29 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:30.460084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10ed56-0759-414a-8f49-2ae53e036f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:30.460667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10ed56-0759-414a-8f49-2ae53e036f7f service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:30.460944 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10ed56-0759-414a-8f49-2ae53e036f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:30.461217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10ed56-0759-414a-8f49-2ae53e036f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:30.471889 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10ed56-0759-414a-8f49-2ae53e036f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:30.473361 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10ed56-0759-414a-8f49-2ae53e036f7f service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:30.484019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1151/2306] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:30.857293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b53dcb-0b8b-4b41-ae72-7407dec12cf8 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:30.858740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-88b53dcb-0b8b-4b41-ae72-7407dec12cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:30.882946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b53dcb-0b8b-4b41-ae72-7407dec12cf8 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:30.889643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:30.890288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:30.890746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:30.891193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] PATH_INFO: `/v3/projects/ef7734b972424fbc9c292640687f88e4` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:30.893157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7734b972424fbc9c292640687f88e4)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:30.895625 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:30.959656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7734b972424fbc9c292640687f88e4', 'request_id': 'req-070c074e-b428-4207-aed0-e128916668d0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.967058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7734b972424fbc9c292640687f88e4', 'request_id': 'req-070c074e-b428-4207-aed0-e128916668d0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.978536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7734b972424fbc9c292640687f88e4', 'request_id': 'req-070c074e-b428-4207-aed0-e128916668d0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.990761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] The token cache is being invalidate because project ef7734b972424fbc9c292640687f88e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:30.991056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.991721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-070c074e-b428-4207-aed0-e128916668d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:30.993354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1156/2307] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:30 2022] DELETE /identity/v3/projects/ef7734b972424fbc9c292640687f88e4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:30.998303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69152d-58f4-48d5-a9a2-8902f2a89a95 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:30.998774 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69152d-58f4-48d5-a9a2-8902f2a89a95 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:30.998867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69152d-58f4-48d5-a9a2-8902f2a89a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:31.019431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-3f69152d-58f4-48d5-a9a2-8902f2a89a95 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:31.022541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-3f69152d-58f4-48d5-a9a2-8902f2a89a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:31.032233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:31.109308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1152/2308] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:31.122613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098df7-a55a-45c2-8565-ba7b77581353 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:31.123240 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098df7-a55a-45c2-8565-ba7b77581353 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:31.124138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098df7-a55a-45c2-8565-ba7b77581353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:31.146800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-3b098df7-a55a-45c2-8565-ba7b77581353 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:31.148362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3b098df7-a55a-45c2-8565-ba7b77581353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:31.157236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1157/2309] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:31.161898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69152d-58f4-48d5-a9a2-8902f2a89a95 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:31.174656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69152d-58f4-48d5-a9a2-8902f2a89a95 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:31.180216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965c24a-4f0d-485f-97e1-2574e3cc4bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:31.181053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965c24a-4f0d-485f-97e1-2574e3cc4bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:31.181434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965c24a-4f0d-485f-97e1-2574e3cc4bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:31.181945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965c24a-4f0d-485f-97e1-2574e3cc4bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:31.182704 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965c24a-4f0d-485f-97e1-2574e3cc4bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:31.183089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965c24a-4f0d-485f-97e1-2574e3cc4bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:31.185034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965c24a-4f0d-485f-97e1-2574e3cc4bc5 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:31.202264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1153/2310] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:48:31.203282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b098df7-a55a-45c2-8565-ba7b77581353 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:31.214678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b098df7-a55a-45c2-8565-ba7b77581353 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:31.221380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790c0e7-e567-4a75-a12c-c87f2ffad52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:31.222302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790c0e7-e567-4a75-a12c-c87f2ffad52a admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:31.222701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790c0e7-e567-4a75-a12c-c87f2ffad52a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:31.222980 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790c0e7-e567-4a75-a12c-c87f2ffad52a admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:31.223626 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790c0e7-e567-4a75-a12c-c87f2ffad52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:31.225692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790c0e7-e567-4a75-a12c-c87f2ffad52a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:31.257504 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:31.270864 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1158/2311] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:31 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:31.272550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965c24a-4f0d-485f-97e1-2574e3cc4bc5 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:31.285266 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965c24a-4f0d-485f-97e1-2574e3cc4bc5 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:31.366500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b11ea6-fb5a-47c2-9540-4d97b5b28293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:31.367754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b11ea6-fb5a-47c2-9540-4d97b5b28293 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:31.367754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b11ea6-fb5a-47c2-9540-4d97b5b28293 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:31.367991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b11ea6-fb5a-47c2-9540-4d97b5b28293 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:31.368738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b11ea6-fb5a-47c2-9540-4d97b5b28293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:31.370898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b11ea6-fb5a-47c2-9540-4d97b5b28293 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:31.398559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1154/2312] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:31 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:31.410334 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790c0e7-e567-4a75-a12c-c87f2ffad52a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:31.424366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790c0e7-e567-4a75-a12c-c87f2ffad52a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:31.429643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870e673-f0b9-46e9-a4be-80a9391ad23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:31.430255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870e673-f0b9-46e9-a4be-80a9391ad23b admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:31.430539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870e673-f0b9-46e9-a4be-80a9391ad23b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:31.430815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870e673-f0b9-46e9-a4be-80a9391ad23b admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:31.431636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870e673-f0b9-46e9-a4be-80a9391ad23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:31.433416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870e673-f0b9-46e9-a4be-80a9391ad23b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:31.453039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1159/2313] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:31.457661 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b11ea6-fb5a-47c2-9540-4d97b5b28293 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:31.468173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b11ea6-fb5a-47c2-9540-4d97b5b28293 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:31.473490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e701ea4-b171-4868-879f-d59e87d3a5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:31.474060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e701ea4-b171-4868-879f-d59e87d3a5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:31.474342 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e701ea4-b171-4868-879f-d59e87d3a5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:31.474618 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e701ea4-b171-4868-879f-d59e87d3a5a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:31.475260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e701ea4-b171-4868-879f-d59e87d3a5a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:31.476920 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e701ea4-b171-4868-879f-d59e87d3a5a5 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:31.492387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1155/2314] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:31.494015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870e673-f0b9-46e9-a4be-80a9391ad23b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:31.506202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870e673-f0b9-46e9-a4be-80a9391ad23b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:31.513836 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1e060-cc3a-4ecd-a933-255a029f9306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:31.514524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1e060-cc3a-4ecd-a933-255a029f9306 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:31.514795 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1e060-cc3a-4ecd-a933-255a029f9306 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:31.515076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1e060-cc3a-4ecd-a933-255a029f9306 admin admin] PATH_INFO: `/v3/projects/76c13daa80f7403298c110265b3539a3/users/182a86c0f20a4bd3bc7523cae7e829b3/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:31.532388 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1e060-cc3a-4ecd-a933-255a029f9306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c13daa80f7403298c110265b3539a3, user_id=182a86c0f20a4bd3bc7523cae7e829b3, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:31.535459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1e060-cc3a-4ecd-a933-255a029f9306 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:31.553222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1160/2315] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:31 2022] PUT /identity/v3/projects/76c13daa80f7403298c110265b3539a3/users/182a86c0f20a4bd3bc7523cae7e829b3/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:31.567091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5ff3d-f628-4b5c-9525-73dc91548fff None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:31.567543 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5ff3d-f628-4b5c-9525-73dc91548fff None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:31.569089 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5ff3d-f628-4b5c-9525-73dc91548fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:31.605913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d0d5ff3d-f628-4b5c-9525-73dc91548fff None None] MFA Rules not processed for user `182a86c0f20a4bd3bc7523cae7e829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:31.607109 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d0d5ff3d-f628-4b5c-9525-73dc91548fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:31.692588 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1156/2316] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:31.973575 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1e060-cc3a-4ecd-a933-255a029f9306 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:31.975565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-afd1e060-cc3a-4ecd-a933-255a029f9306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:32.032577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1e060-cc3a-4ecd-a933-255a029f9306 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:32.036123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb02479-e263-49dd-bef6-c6756955b733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:32.036764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb02479-e263-49dd-bef6-c6756955b733 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:32.037046 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb02479-e263-49dd-bef6-c6756955b733 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:32.037324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb02479-e263-49dd-bef6-c6756955b733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:32.045580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb02479-e263-49dd-bef6-c6756955b733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:32.047025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb02479-e263-49dd-bef6-c6756955b733 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:32.055858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1161/2317] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:34.799260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451f0c5-16e7-4694-8344-47a429407f88 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:34.799760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451f0c5-16e7-4694-8344-47a429407f88 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:34.799845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451f0c5-16e7-4694-8344-47a429407f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:34.821311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b451f0c5-16e7-4694-8344-47a429407f88 None None] MFA Rules not processed for user `182a86c0f20a4bd3bc7523cae7e829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:34.822607 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b451f0c5-16e7-4694-8344-47a429407f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:34.830202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1157/2318] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:40.386465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb02479-e263-49dd-bef6-c6756955b733 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:40.387937 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5eb02479-e263-49dd-bef6-c6756955b733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:40.456275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb02479-e263-49dd-bef6-c6756955b733 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:40.460941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:40.461657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:40.461997 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:40.462335 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] PATH_INFO: `/v3/users/296f2991d00c4a17b30298d9a7dc8bf1` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:40.463540 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] RBAC: Authorizing `identity:delete_user(user_id=296f2991d00c4a17b30298d9a7dc8bf1)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:40.467841 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:40.525237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '296f2991d00c4a17b30298d9a7dc8bf1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.533482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '296f2991d00c4a17b30298d9a7dc8bf1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.534190 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '296f2991d00c4a17b30298d9a7dc8bf1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.534791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '296f2991d00c4a17b30298d9a7dc8bf1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.550060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25559df1-79bd-4b98-b380-9f5a5385954d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '296f2991d00c4a17b30298d9a7dc8bf1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.555515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:40.562831 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451f0c5-16e7-4694-8344-47a429407f88 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:40.564708 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b451f0c5-16e7-4694-8344-47a429407f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:40.573343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1162/2319] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:40 2022] DELETE /identity/v3/users/296f2991d00c4a17b30298d9a7dc8bf1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:40.574842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:40.577740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876f759-84e2-42f4-ab8e-beaea23ed0a2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:40.578026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876f759-84e2-42f4-ab8e-beaea23ed0a2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:40.578301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876f759-84e2-42f4-ab8e-beaea23ed0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:40.581383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451f0c5-16e7-4694-8344-47a429407f88 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:40.600177 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-4876f759-84e2-42f4-ab8e-beaea23ed0a2 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:40.601580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4876f759-84e2-42f4-ab8e-beaea23ed0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:40.654080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:40.655148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:40.655435 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:40.655717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] PATH_INFO: `/v3/users/38375e7dc8504f898c25c1fcdf14f89e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:40.657180 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38375e7dc8504f898c25c1fcdf14f89e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:40.659971 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:40.671574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1163/2320] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:40.692197 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876f759-84e2-42f4-ab8e-beaea23ed0a2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:40.693969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4876f759-84e2-42f4-ab8e-beaea23ed0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:40.721659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38375e7dc8504f898c25c1fcdf14f89e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.731808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38375e7dc8504f898c25c1fcdf14f89e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.732659 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38375e7dc8504f898c25c1fcdf14f89e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.733549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38375e7dc8504f898c25c1fcdf14f89e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.744381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-210e39dc-6905-48f1-96ba-a519c3da3d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38375e7dc8504f898c25c1fcdf14f89e'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:40.770859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1158/2321] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:40 2022] DELETE /identity/v3/users/38375e7dc8504f898c25c1fcdf14f89e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:40.775175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932d1c-b2b3-495d-a226-89bb775a78e3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:40.775529 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932d1c-b2b3-495d-a226-89bb775a78e3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:40.775873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932d1c-b2b3-495d-a226-89bb775a78e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:40.779512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876f759-84e2-42f4-ab8e-beaea23ed0a2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:40.807211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-59932d1c-b2b3-495d-a226-89bb775a78e3 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:40.810101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-59932d1c-b2b3-495d-a226-89bb775a78e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:40.846790 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:40.847701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:40.848239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:40.848759 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:40.851154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:40.901380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1159/2322] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:40.911959 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59932d1c-b2b3-495d-a226-89bb775a78e3 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:40.913451 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-59932d1c-b2b3-495d-a226-89bb775a78e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:40.921676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:40.927673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59932d1c-b2b3-495d-a226-89bb775a78e3 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:40.937579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2731a-88d9-4a0c-9236-0929af9840c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:40.938366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2731a-88d9-4a0c-9236-0929af9840c1 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:40.938652 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2731a-88d9-4a0c-9236-0929af9840c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:40.938912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:40.939083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2731a-88d9-4a0c-9236-0929af9840c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:40.940537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:40.951358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1164/2323] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:40.953946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2731a-88d9-4a0c-9236-0929af9840c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:40.955644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2731a-88d9-4a0c-9236-0929af9840c1 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:40.966561 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1160/2324] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:41.265764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.267829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:41.299195 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5d13b-d548-48a6-9d5d-6599e30b059e service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:41.306457 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:41.307166 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.307958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.307958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] PATH_INFO: `/v3/projects/c4539f07e24f42e18507a1cc54020231` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.309837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4539f07e24f42e18507a1cc54020231)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:41.312060 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:41.395404 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4539f07e24f42e18507a1cc54020231', 'request_id': 'req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.402231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4539f07e24f42e18507a1cc54020231', 'request_id': 'req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.410874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4539f07e24f42e18507a1cc54020231', 'request_id': 'req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.431328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] The token cache is being invalidate because project c4539f07e24f42e18507a1cc54020231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:41.431628 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.432542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-424ce5d0-f4a9-4112-889a-0ba8f2c8eff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.434601 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1165/2325] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:41 2022] DELETE /identity/v3/projects/c4539f07e24f42e18507a1cc54020231 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:41.439772 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebcd83-76f7-4c25-bf0e-82657a5f0816 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.440060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebcd83-76f7-4c25-bf0e-82657a5f0816 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.440372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebcd83-76f7-4c25-bf0e-82657a5f0816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.461898 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-47ebcd83-76f7-4c25-bf0e-82657a5f0816 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:41.463127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-47ebcd83-76f7-4c25-bf0e-82657a5f0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:41.469322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:41.552472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1161/2326] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:41.561092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3f4cf-5731-4ec8-9621-a4a3a9795445 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.561697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3f4cf-5731-4ec8-9621-a4a3a9795445 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.561839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebcd83-76f7-4c25-bf0e-82657a5f0816 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.562247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3f4cf-5731-4ec8-9621-a4a3a9795445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.563467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-47ebcd83-76f7-4c25-bf0e-82657a5f0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:41.576038 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebcd83-76f7-4c25-bf0e-82657a5f0816 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:41.581829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:41.582561 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.584662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-8ba3f4cf-5731-4ec8-9621-a4a3a9795445 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:41.585865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-8ba3f4cf-5731-4ec8-9621-a4a3a9795445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:41.588260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.588671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] PATH_INFO: `/v3/projects/1e6f184a37f64d58a055953fad43ce9a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.590854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6f184a37f64d58a055953fad43ce9a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:41.596387 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1166/2327] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:41.596539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:41.598935 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3f4cf-5731-4ec8-9621-a4a3a9795445 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.608510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3f4cf-5731-4ec8-9621-a4a3a9795445 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:41.614765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1259af-8ddb-4a85-a8d5-a836f936ae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:41.615362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1259af-8ddb-4a85-a8d5-a836f936ae5a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.615644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1259af-8ddb-4a85-a8d5-a836f936ae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.619564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1259af-8ddb-4a85-a8d5-a836f936ae5a admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.620762 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1259af-8ddb-4a85-a8d5-a836f936ae5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:41.621070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1259af-8ddb-4a85-a8d5-a836f936ae5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:41.622694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1259af-8ddb-4a85-a8d5-a836f936ae5a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:41.635169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1167/2328] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:48:41.639473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1259af-8ddb-4a85-a8d5-a836f936ae5a admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.649369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1259af-8ddb-4a85-a8d5-a836f936ae5a admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:41.653309 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6f184a37f64d58a055953fad43ce9a', 'request_id': 'req-19adbf67-a0ed-4c14-b1ba-3b1478145b66'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.662721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6f184a37f64d58a055953fad43ce9a', 'request_id': 'req-19adbf67-a0ed-4c14-b1ba-3b1478145b66'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.670839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6f184a37f64d58a055953fad43ce9a', 'request_id': 'req-19adbf67-a0ed-4c14-b1ba-3b1478145b66'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.684463 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] The token cache is being invalidate because project 1e6f184a37f64d58a055953fad43ce9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:41.684749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.685416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-19adbf67-a0ed-4c14-b1ba-3b1478145b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:41.687392 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1162/2329] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:41 2022] DELETE /identity/v3/projects/1e6f184a37f64d58a055953fad43ce9a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:41.692021 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56490137-6000-42c1-ba51-bb6aecbbf230 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.692360 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56490137-6000-42c1-ba51-bb6aecbbf230 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.692907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56490137-6000-42c1-ba51-bb6aecbbf230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.718261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-56490137-6000-42c1-ba51-bb6aecbbf230 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:41.720119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-56490137-6000-42c1-ba51-bb6aecbbf230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:41.723397 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c356-953f-4541-a01d-7a46b603ead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:41.724010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c356-953f-4541-a01d-7a46b603ead3 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.724332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c356-953f-4541-a01d-7a46b603ead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.724657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c356-953f-4541-a01d-7a46b603ead3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.725905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036c356-953f-4541-a01d-7a46b603ead3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:41.728110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036c356-953f-4541-a01d-7a46b603ead3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:41.730210 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1163/2330] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:48:41.738568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc84bbc-2089-4959-a5e0-3d5ce5d8c27d None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.738854 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc84bbc-2089-4959-a5e0-3d5ce5d8c27d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.739123 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc84bbc-2089-4959-a5e0-3d5ce5d8c27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.759791 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-4dc84bbc-2089-4959-a5e0-3d5ce5d8c27d None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:41.760925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-4dc84bbc-2089-4959-a5e0-3d5ce5d8c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:41.763299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1168/2331] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:41 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:41.764152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c356-953f-4541-a01d-7a46b603ead3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.766131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a036c356-953f-4541-a01d-7a46b603ead3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:41.821296 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:41.832241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1164/2332] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:48:41.833398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc84bbc-2089-4959-a5e0-3d5ce5d8c27d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.842692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc84bbc-2089-4959-a5e0-3d5ce5d8c27d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:41.847953 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c65ae-5c62-41d7-b55c-ec883346c267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:41.848611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c65ae-5c62-41d7-b55c-ec883346c267 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.848894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c65ae-5c62-41d7-b55c-ec883346c267 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.849333 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c65ae-5c62-41d7-b55c-ec883346c267 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.849987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c65ae-5c62-41d7-b55c-ec883346c267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:41.850295 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c65ae-5c62-41d7-b55c-ec883346c267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:41.851900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c65ae-5c62-41d7-b55c-ec883346c267 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:41.862084 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c356-953f-4541-a01d-7a46b603ead3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:41.865403 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1165/2333] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 14:48:41.866879 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89177c35-2d95-4f69-a22f-b2e66ab2ce8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:41.867488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89177c35-2d95-4f69-a22f-b2e66ab2ce8b admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.867765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89177c35-2d95-4f69-a22f-b2e66ab2ce8b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.868036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89177c35-2d95-4f69-a22f-b2e66ab2ce8b admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.868705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89177c35-2d95-4f69-a22f-b2e66ab2ce8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:41.870781 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89177c35-2d95-4f69-a22f-b2e66ab2ce8b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:41.871251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c65ae-5c62-41d7-b55c-ec883346c267 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.882068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c65ae-5c62-41d7-b55c-ec883346c267 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:41.887214 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69121426-91aa-4bff-a7f9-962515aae090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:41.887910 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69121426-91aa-4bff-a7f9-962515aae090 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.888223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69121426-91aa-4bff-a7f9-962515aae090 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.888554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69121426-91aa-4bff-a7f9-962515aae090 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.889190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69121426-91aa-4bff-a7f9-962515aae090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:41.891200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69121426-91aa-4bff-a7f9-962515aae090 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:41.897565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1169/2334] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:41 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:41.901604 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89177c35-2d95-4f69-a22f-b2e66ab2ce8b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.909845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:41.914757 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89177c35-2d95-4f69-a22f-b2e66ab2ce8b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:41.919782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486a548-4485-4e27-a1a6-5b4b094d6f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:41.920424 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486a548-4485-4e27-a1a6-5b4b094d6f02 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:41.920706 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486a548-4485-4e27-a1a6-5b4b094d6f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:41.920980 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486a548-4485-4e27-a1a6-5b4b094d6f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:41.921618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486a548-4485-4e27-a1a6-5b4b094d6f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:41.923564 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486a548-4485-4e27-a1a6-5b4b094d6f02 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:41.931967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1166/2335] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:41 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:41.936478 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1170/2336] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:41.938666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486a548-4485-4e27-a1a6-5b4b094d6f02 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.939156 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69121426-91aa-4bff-a7f9-962515aae090 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:41.950529 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486a548-4485-4e27-a1a6-5b4b094d6f02 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.010195 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69121426-91aa-4bff-a7f9-962515aae090 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.015606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf051aa-350f-45e5-a156-23b59db57632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.017192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf051aa-350f-45e5-a156-23b59db57632 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.017535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf051aa-350f-45e5-a156-23b59db57632 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.017867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf051aa-350f-45e5-a156-23b59db57632 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.018504 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf051aa-350f-45e5-a156-23b59db57632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.020460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf051aa-350f-45e5-a156-23b59db57632 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.022036 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52ba3-8a29-4e57-a39e-47dbaa623b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.022798 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52ba3-8a29-4e57-a39e-47dbaa623b17 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.023106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52ba3-8a29-4e57-a39e-47dbaa623b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.023509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52ba3-8a29-4e57-a39e-47dbaa623b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.024226 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df52ba3-8a29-4e57-a39e-47dbaa623b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.025946 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df52ba3-8a29-4e57-a39e-47dbaa623b17 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.038526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1171/2337] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:42.042720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52ba3-8a29-4e57-a39e-47dbaa623b17 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.052111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1167/2338] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:41 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:42.053486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf051aa-350f-45e5-a156-23b59db57632 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.055750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52ba3-8a29-4e57-a39e-47dbaa623b17 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.061102 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e6e47-69a1-4833-9c45-8d905b9feff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.061978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e6e47-69a1-4833-9c45-8d905b9feff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.062443 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e6e47-69a1-4833-9c45-8d905b9feff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.062750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e6e47-69a1-4833-9c45-8d905b9feff7 admin admin] PATH_INFO: `/v3/projects/34676c921ba04710b14cfa94eab069d3/users/36a281f7f9c8435c8ad65ae4b0bf141a/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.069820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf051aa-350f-45e5-a156-23b59db57632 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.074987 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc217550-0e73-42cb-8d72-45e5d0817f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.075751 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc217550-0e73-42cb-8d72-45e5d0817f25 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.076528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc217550-0e73-42cb-8d72-45e5d0817f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.076528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc217550-0e73-42cb-8d72-45e5d0817f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.077364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc217550-0e73-42cb-8d72-45e5d0817f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.079502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc217550-0e73-42cb-8d72-45e5d0817f25 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.085668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e6e47-69a1-4833-9c45-8d905b9feff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34676c921ba04710b14cfa94eab069d3, user_id=36a281f7f9c8435c8ad65ae4b0bf141a, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.089365 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e6e47-69a1-4833-9c45-8d905b9feff7 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.093490 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1168/2339] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:42.097554 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc217550-0e73-42cb-8d72-45e5d0817f25 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.113655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1172/2340] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:42 2022] PUT /identity/v3/projects/34676c921ba04710b14cfa94eab069d3/users/36a281f7f9c8435c8ad65ae4b0bf141a/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:42.113845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc217550-0e73-42cb-8d72-45e5d0817f25 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.119671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f893f-9876-49b4-9831-a5b3c4a3b3a6 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.120092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f893f-9876-49b4-9831-a5b3c4a3b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.120491 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f893f-9876-49b4-9831-a5b3c4a3b3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.143518 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-7e0f893f-9876-49b4-9831-a5b3c4a3b3a6 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:42.145029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7e0f893f-9876-49b4-9831-a5b3c4a3b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:42.196061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a438b-32e2-442c-84b6-8a7150d87842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.196061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a438b-32e2-442c-84b6-8a7150d87842 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.196433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a438b-32e2-442c-84b6-8a7150d87842 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.196719 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a438b-32e2-442c-84b6-8a7150d87842 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.197366 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a438b-32e2-442c-84b6-8a7150d87842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.199034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a438b-32e2-442c-84b6-8a7150d87842 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.212575 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1169/2341] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:42.216673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a438b-32e2-442c-84b6-8a7150d87842 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.228804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1173/2342] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:42.230715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a438b-32e2-442c-84b6-8a7150d87842 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.237117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc63ae0-a653-4384-aea4-b1c561198983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.238285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63ae0-a653-4384-aea4-b1c561198983 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.238566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63ae0-a653-4384-aea4-b1c561198983 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.238843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63ae0-a653-4384-aea4-b1c561198983 admin admin] PATH_INFO: `/v3/projects/e805da7ed5a84d469a2a8c3f5bbc95aa/users/e11837cf9f73454eb81b5c1de6192f3c/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.241037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f893f-9876-49b4-9831-a5b3c4a3b3a6 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.243435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7e0f893f-9876-49b4-9831-a5b3c4a3b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:42.269048 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc63ae0-a653-4384-aea4-b1c561198983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e805da7ed5a84d469a2a8c3f5bbc95aa, user_id=e11837cf9f73454eb81b5c1de6192f3c, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.272530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc63ae0-a653-4384-aea4-b1c561198983 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.286186 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:42.290024 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1170/2343] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:42 2022] PUT /identity/v3/projects/e805da7ed5a84d469a2a8c3f5bbc95aa/users/e11837cf9f73454eb81b5c1de6192f3c/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:42.304885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834174fc-3513-4738-b06f-ec6fd2724e76 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.305237 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834174fc-3513-4738-b06f-ec6fd2724e76 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.305885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834174fc-3513-4738-b06f-ec6fd2724e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.317746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f893f-9876-49b4-9831-a5b3c4a3b3a6 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.338839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-834174fc-3513-4738-b06f-ec6fd2724e76 None None] MFA Rules not processed for user `e11837cf9f73454eb81b5c1de6192f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:42.340055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-834174fc-3513-4738-b06f-ec6fd2724e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:42.376858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913b005-b2b5-4b69-aa81-e79d007df39b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.377444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913b005-b2b5-4b69-aa81-e79d007df39b service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.377958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913b005-b2b5-4b69-aa81-e79d007df39b service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.378152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913b005-b2b5-4b69-aa81-e79d007df39b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.390620 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:42.436493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1171/2344] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:48:42.438870 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834174fc-3513-4738-b06f-ec6fd2724e76 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.451973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913b005-b2b5-4b69-aa81-e79d007df39b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.453640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913b005-b2b5-4b69-aa81-e79d007df39b service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.464135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1174/2345] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:42.514275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834174fc-3513-4738-b06f-ec6fd2724e76 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.520249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefac59-ec05-4e65-858b-9930e14939a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.520899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefac59-ec05-4e65-858b-9930e14939a2 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.521263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefac59-ec05-4e65-858b-9930e14939a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.521597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefac59-ec05-4e65-858b-9930e14939a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.522523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefac59-ec05-4e65-858b-9930e14939a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.524495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefac59-ec05-4e65-858b-9930e14939a2 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.549024 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913b005-b2b5-4b69-aa81-e79d007df39b service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.551165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2913b005-b2b5-4b69-aa81-e79d007df39b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:42.566972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913b005-b2b5-4b69-aa81-e79d007df39b service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.568172 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1172/2346] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:42 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:42.573205 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefac59-ec05-4e65-858b-9930e14939a2 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.645076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.645710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.646876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.646876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] PATH_INFO: `/v3/users/717d38be0fed4071953fdf4a1aa9f198` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.647615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] RBAC: Authorizing `identity:delete_user(user_id=717d38be0fed4071953fdf4a1aa9f198)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.649650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.660883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefac59-ec05-4e65-858b-9930e14939a2 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.669442 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa376c20-bf24-4c47-b39a-5d9844f24375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.670276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa376c20-bf24-4c47-b39a-5d9844f24375 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.670714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa376c20-bf24-4c47-b39a-5d9844f24375 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.671143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa376c20-bf24-4c47-b39a-5d9844f24375 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.672074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa376c20-bf24-4c47-b39a-5d9844f24375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.676370 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa376c20-bf24-4c47-b39a-5d9844f24375 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.699348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:42.704767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1173/2347] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:42 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:42.707393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717d38be0fed4071953fdf4a1aa9f198'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:42.710731 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa376c20-bf24-4c47-b39a-5d9844f24375 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.715967 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717d38be0fed4071953fdf4a1aa9f198'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:42.717167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717d38be0fed4071953fdf4a1aa9f198'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:42.717920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717d38be0fed4071953fdf4a1aa9f198'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:42.729749 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa376c20-bf24-4c47-b39a-5d9844f24375 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.730222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-50c3a4a7-f663-4b29-b330-7ae367c4a38e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717d38be0fed4071953fdf4a1aa9f198'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:42.736980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059663c-4a3e-4726-bc33-1b8b3c44b072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.737865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059663c-4a3e-4726-bc33-1b8b3c44b072 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.738058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059663c-4a3e-4726-bc33-1b8b3c44b072 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.738393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059663c-4a3e-4726-bc33-1b8b3c44b072 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.739049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059663c-4a3e-4726-bc33-1b8b3c44b072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.740700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059663c-4a3e-4726-bc33-1b8b3c44b072 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.753311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1174/2348] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:42.754788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1175/2349] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:42 2022] DELETE /identity/v3/users/717d38be0fed4071953fdf4a1aa9f198 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:42.757387 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059663c-4a3e-4726-bc33-1b8b3c44b072 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.758900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f218d2-2cce-475b-99cc-61afdd087ebe None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.759188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f218d2-2cce-475b-99cc-61afdd087ebe None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.759463 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f218d2-2cce-475b-99cc-61afdd087ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.762945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d059663c-4a3e-4726-bc33-1b8b3c44b072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:42.777459 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059663c-4a3e-4726-bc33-1b8b3c44b072 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.780546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-05f218d2-2cce-475b-99cc-61afdd087ebe None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:42.781912 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-05f218d2-2cce-475b-99cc-61afdd087ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:42.820640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:42.853136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719900c1-50f7-4241-86e2-d47c78427919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.853857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719900c1-50f7-4241-86e2-d47c78427919 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.854192 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719900c1-50f7-4241-86e2-d47c78427919 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.854540 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719900c1-50f7-4241-86e2-d47c78427919 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.855237 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719900c1-50f7-4241-86e2-d47c78427919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.857346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719900c1-50f7-4241-86e2-d47c78427919 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.858973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1176/2350] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:42.871797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1175/2351] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:42.872090 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f218d2-2cce-475b-99cc-61afdd087ebe None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.872891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-05f218d2-2cce-475b-99cc-61afdd087ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:42.873055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719900c1-50f7-4241-86e2-d47c78427919 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:42.884071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719900c1-50f7-4241-86e2-d47c78427919 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.889484 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929d859-2fc2-41cd-b5e2-8a0bf21340fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:42.890245 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929d859-2fc2-41cd-b5e2-8a0bf21340fe admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.890637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929d859-2fc2-41cd-b5e2-8a0bf21340fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.891644 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929d859-2fc2-41cd-b5e2-8a0bf21340fe admin admin] PATH_INFO: `/v3/projects/a8a726ebbf624cea85440e642d0b6ec6/users/49d7de7930ba44b8ad663c13dfdd0782/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.932813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929d859-2fc2-41cd-b5e2-8a0bf21340fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a726ebbf624cea85440e642d0b6ec6, user_id=49d7de7930ba44b8ad663c13dfdd0782, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:42.935485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929d859-2fc2-41cd-b5e2-8a0bf21340fe admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:42.951904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1176/2352] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:42 2022] PUT /identity/v3/projects/a8a726ebbf624cea85440e642d0b6ec6/users/49d7de7930ba44b8ad663c13dfdd0782/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:42.955199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f218d2-2cce-475b-99cc-61afdd087ebe None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:42.963376 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11578e3-4a4e-4b75-8e60-4db3db2357fe None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:42.963654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11578e3-4a4e-4b75-8e60-4db3db2357fe None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:42.963928 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11578e3-4a4e-4b75-8e60-4db3db2357fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:42.996291 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d11578e3-4a4e-4b75-8e60-4db3db2357fe None None] MFA Rules not processed for user `49d7de7930ba44b8ad663c13dfdd0782`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:42.998691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d11578e3-4a4e-4b75-8e60-4db3db2357fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:43.011422 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffab6ee-b83e-4dee-b9e1-df01f352f598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.012047 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffab6ee-b83e-4dee-b9e1-df01f352f598 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.012328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffab6ee-b83e-4dee-b9e1-df01f352f598 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.012633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffab6ee-b83e-4dee-b9e1-df01f352f598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.051637 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:43.100449 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1177/2353] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:48:43.109077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffab6ee-b83e-4dee-b9e1-df01f352f598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.111165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffab6ee-b83e-4dee-b9e1-df01f352f598 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.129791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1177/2354] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:43.163862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11578e3-4a4e-4b75-8e60-4db3db2357fe None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.175756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11578e3-4a4e-4b75-8e60-4db3db2357fe None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.180972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccb91f-e02d-41c2-bfe7-679ef7ef80b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.181482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccb91f-e02d-41c2-bfe7-679ef7ef80b4 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.181939 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccb91f-e02d-41c2-bfe7-679ef7ef80b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.182074 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccb91f-e02d-41c2-bfe7-679ef7ef80b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.182938 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccb91f-e02d-41c2-bfe7-679ef7ef80b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.185126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccb91f-e02d-41c2-bfe7-679ef7ef80b4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.220397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1178/2355] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:43 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:43.224984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffab6ee-b83e-4dee-b9e1-df01f352f598 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.237790 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffab6ee-b83e-4dee-b9e1-df01f352f598 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.311389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd80997-8743-4b1f-83b4-558593698007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.312377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd80997-8743-4b1f-83b4-558593698007 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.312506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd80997-8743-4b1f-83b4-558593698007 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.312838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd80997-8743-4b1f-83b4-558593698007 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.313511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd80997-8743-4b1f-83b4-558593698007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.315921 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd80997-8743-4b1f-83b4-558593698007 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.343447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1178/2356] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:43 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:43.345303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccb91f-e02d-41c2-bfe7-679ef7ef80b4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.359348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccb91f-e02d-41c2-bfe7-679ef7ef80b4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.366267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350354-5bb2-4c84-9e5c-60cbc48a2fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.366267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350354-5bb2-4c84-9e5c-60cbc48a2fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.366574 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350354-5bb2-4c84-9e5c-60cbc48a2fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.366965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350354-5bb2-4c84-9e5c-60cbc48a2fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.367894 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350354-5bb2-4c84-9e5c-60cbc48a2fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.369551 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350354-5bb2-4c84-9e5c-60cbc48a2fb9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.382225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1179/2357] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:43.386460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd80997-8743-4b1f-83b4-558593698007 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.400562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd80997-8743-4b1f-83b4-558593698007 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.405983 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0449e39-b4d1-4614-b825-4014315153c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.406844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0449e39-b4d1-4614-b825-4014315153c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.407671 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0449e39-b4d1-4614-b825-4014315153c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.407958 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0449e39-b4d1-4614-b825-4014315153c6 admin admin] PATH_INFO: `/v3/projects/4d73256f1b7747dba53a039c729c2998/users/46f694bf8e044c4a8979301762348fcf/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.418455 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2ddc4-e63a-4151-bc05-b65d16274ed5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.418763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2ddc4-e63a-4151-bc05-b65d16274ed5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.419031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2ddc4-e63a-4151-bc05-b65d16274ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.437372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0449e39-b4d1-4614-b825-4014315153c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d73256f1b7747dba53a039c729c2998, user_id=46f694bf8e044c4a8979301762348fcf, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.441257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0449e39-b4d1-4614-b825-4014315153c6 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.458460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-83f2ddc4-e63a-4151-bc05-b65d16274ed5 None None] MFA Rules not processed for user `36a281f7f9c8435c8ad65ae4b0bf141a`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:43.459853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-83f2ddc4-e63a-4151-bc05-b65d16274ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:43.462908 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1179/2358] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:43 2022] PUT /identity/v3/projects/4d73256f1b7747dba53a039c729c2998/users/46f694bf8e044c4a8979301762348fcf/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:43.467291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0449e39-b4d1-4614-b825-4014315153c6 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.554170 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0449e39-b4d1-4614-b825-4014315153c6 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.559002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02587446-be73-4a3c-b5c6-6b742ab4824e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.559607 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02587446-be73-4a3c-b5c6-6b742ab4824e admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.559886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02587446-be73-4a3c-b5c6-6b742ab4824e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.560159 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02587446-be73-4a3c-b5c6-6b742ab4824e admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.560838 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02587446-be73-4a3c-b5c6-6b742ab4824e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.561298 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:43.562441 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02587446-be73-4a3c-b5c6-6b742ab4824e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.573481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1180/2359] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:43.574204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02587446-be73-4a3c-b5c6-6b742ab4824e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.575971 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-02587446-be73-4a3c-b5c6-6b742ab4824e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:43.581796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1180/2360] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:48:43.582795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2ddc4-e63a-4151-bc05-b65d16274ed5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.589995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02587446-be73-4a3c-b5c6-6b742ab4824e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.595500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.596140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.596490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.596745 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] PATH_INFO: `/v3/projects/9b4493ebf93b4305bf4e90c5d36013c5` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.598075 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2ddc4-e63a-4151-bc05-b65d16274ed5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.598385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4493ebf93b4305bf4e90c5d36013c5)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.602362 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.603733 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a001ef-d110-4a77-887f-230b446a9edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.604369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a001ef-d110-4a77-887f-230b446a9edf admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.604648 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a001ef-d110-4a77-887f-230b446a9edf admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.605095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a001ef-d110-4a77-887f-230b446a9edf admin admin] PATH_INFO: `/v3/projects/4d73256f1b7747dba53a039c729c2998/users/46f694bf8e044c4a8979301762348fcf/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.607329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a001ef-d110-4a77-887f-230b446a9edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d73256f1b7747dba53a039c729c2998, user_id=46f694bf8e044c4a8979301762348fcf, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.611794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a001ef-d110-4a77-887f-230b446a9edf admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.633313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1181/2361] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:43 2022] PUT /identity/v3/projects/4d73256f1b7747dba53a039c729c2998/users/46f694bf8e044c4a8979301762348fcf/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:43.633313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a001ef-d110-4a77-887f-230b446a9edf admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.633673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:43.635842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b8a001ef-d110-4a77-887f-230b446a9edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:43.662547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4493ebf93b4305bf4e90c5d36013c5', 'request_id': 'req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:43.668009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4493ebf93b4305bf4e90c5d36013c5', 'request_id': 'req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:43.676064 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4493ebf93b4305bf4e90c5d36013c5', 'request_id': 'req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:43.688333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] The token cache is being invalidate because project 9b4493ebf93b4305bf4e90c5d36013c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:43.688650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:43.689311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-dd906cf5-daa7-4db8-8e0a-3726d37ebde0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:43.690810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1181/2362] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:43 2022] DELETE /identity/v3/projects/9b4493ebf93b4305bf4e90c5d36013c5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:43.693734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edf7b1-be5f-4913-b39e-e8fec59490e2 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.694907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edf7b1-be5f-4913-b39e-e8fec59490e2 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.694907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edf7b1-be5f-4913-b39e-e8fec59490e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.699501 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a001ef-d110-4a77-887f-230b446a9edf admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.726795 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-a7edf7b1-be5f-4913-b39e-e8fec59490e2 None None] MFA Rules not processed for user `46f694bf8e044c4a8979301762348fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:43.728609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a7edf7b1-be5f-4913-b39e-e8fec59490e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:43.756533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.757153 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.757434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.757705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.759433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:43.805563 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1182/2363] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:43.810216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edf7b1-be5f-4913-b39e-e8fec59490e2 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.811897 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a7edf7b1-be5f-4913-b39e-e8fec59490e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:43.829930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.832766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.847267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1182/2364] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:43.848175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:43.849899 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:43.865101 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9a2ca-897f-4b2b-9c99-d216073e84a8 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.871354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9853b0d-0f87-4862-9f9e-92a6fdd9c325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.873072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9853b0d-0f87-4862-9f9e-92a6fdd9c325 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.873461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9853b0d-0f87-4862-9f9e-92a6fdd9c325 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.874090 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9853b0d-0f87-4862-9f9e-92a6fdd9c325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.877877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f9853b0d-0f87-4862-9f9e-92a6fdd9c325 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:43.890034 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edf7b1-be5f-4913-b39e-e8fec59490e2 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:43.901566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:43.902222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:43.902506 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:43.902791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] PATH_INFO: `/v3/users/40833e9a11de42a5a0b4c6f460763f9a` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:43.903951 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] RBAC: Authorizing `identity:delete_user(user_id=40833e9a11de42a5a0b4c6f460763f9a)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:43.906386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:43.979700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:43.990177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:44.012834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40833e9a11de42a5a0b4c6f460763f9a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.013292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9853b0d-0f87-4862-9f9e-92a6fdd9c325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:44.015181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9853b0d-0f87-4862-9f9e-92a6fdd9c325 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:44.027041 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40833e9a11de42a5a0b4c6f460763f9a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.027804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40833e9a11de42a5a0b4c6f460763f9a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.029054 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40833e9a11de42a5a0b4c6f460763f9a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.030670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1183/2365] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:44.039311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c8d20e86-6208-4074-9ad7-6c9ba5c8b49f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40833e9a11de42a5a0b4c6f460763f9a'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.064281 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1183/2366] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:43 2022] DELETE /identity/v3/users/40833e9a11de42a5a0b4c6f460763f9a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:44.484162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9853b0d-0f87-4862-9f9e-92a6fdd9c325 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:44.511222 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9853b0d-0f87-4862-9f9e-92a6fdd9c325 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:44.575541 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:44.576103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:44.576400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:44.576679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] PATH_INFO: `/v3/projects/ccce311da4f64971b88759cc7e5d9baa` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:44.577788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccce311da4f64971b88759cc7e5d9baa)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:44.579913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:44.628670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccce311da4f64971b88759cc7e5d9baa', 'request_id': 'req-9e4bf16e-a094-480d-aa2e-41671ab06491'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.634084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccce311da4f64971b88759cc7e5d9baa', 'request_id': 'req-9e4bf16e-a094-480d-aa2e-41671ab06491'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.641593 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccce311da4f64971b88759cc7e5d9baa', 'request_id': 'req-9e4bf16e-a094-480d-aa2e-41671ab06491'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.652663 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] The token cache is being invalidate because project ccce311da4f64971b88759cc7e5d9baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:44.652954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.653581 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-9e4bf16e-a094-480d-aa2e-41671ab06491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:44.655039 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1184/2367] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:44 2022] DELETE /identity/v3/projects/ccce311da4f64971b88759cc7e5d9baa => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:44.660860 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba72b25-d3b0-4617-9953-f5010d3bd4d4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:44.661321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba72b25-d3b0-4617-9953-f5010d3bd4d4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:44.661453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba72b25-d3b0-4617-9953-f5010d3bd4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:44.680844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-6ba72b25-d3b0-4617-9953-f5010d3bd4d4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:44.682029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6ba72b25-d3b0-4617-9953-f5010d3bd4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:44.741894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1184/2368] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:44.749519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4655b-9824-42ca-a2f6-3ef85d4f3acf None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:44.749804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4655b-9824-42ca-a2f6-3ef85d4f3acf None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:44.750083 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4655b-9824-42ca-a2f6-3ef85d4f3acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:44.770340 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-91e4655b-9824-42ca-a2f6-3ef85d4f3acf None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:44.771605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-91e4655b-9824-42ca-a2f6-3ef85d4f3acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:44.779478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1185/2369] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:44.783835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba72b25-d3b0-4617-9953-f5010d3bd4d4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:44.793960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba72b25-d3b0-4617-9953-f5010d3bd4d4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:44.798531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169310f-1869-4b9e-a684-638b8cfd4d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:44.799124 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169310f-1869-4b9e-a684-638b8cfd4d38 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:44.799399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169310f-1869-4b9e-a684-638b8cfd4d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:44.799667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169310f-1869-4b9e-a684-638b8cfd4d38 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:44.800329 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169310f-1869-4b9e-a684-638b8cfd4d38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:44.800658 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169310f-1869-4b9e-a684-638b8cfd4d38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:44.802275 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169310f-1869-4b9e-a684-638b8cfd4d38 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:44.814511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1185/2370] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:48:44.818748 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4655b-9824-42ca-a2f6-3ef85d4f3acf None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:44.827824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4655b-9824-42ca-a2f6-3ef85d4f3acf None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:44.833142 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e23229-095b-44ad-b8ca-fa7e1ac40488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:44.833729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23229-095b-44ad-b8ca-fa7e1ac40488 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:44.834005 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23229-095b-44ad-b8ca-fa7e1ac40488 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:44.834275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23229-095b-44ad-b8ca-fa7e1ac40488 admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:44.834904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e23229-095b-44ad-b8ca-fa7e1ac40488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:44.836839 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e23229-095b-44ad-b8ca-fa7e1ac40488 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:44.863900 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1186/2371] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:44 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:44.868096 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169310f-1869-4b9e-a684-638b8cfd4d38 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:44.878782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169310f-1869-4b9e-a684-638b8cfd4d38 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:44.933814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91769953-ddbd-4536-9bfc-f82cd67084e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:44.934413 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91769953-ddbd-4536-9bfc-f82cd67084e7 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:44.934699 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91769953-ddbd-4536-9bfc-f82cd67084e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:44.934976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91769953-ddbd-4536-9bfc-f82cd67084e7 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:44.935611 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91769953-ddbd-4536-9bfc-f82cd67084e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:44.937593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91769953-ddbd-4536-9bfc-f82cd67084e7 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:44.958528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1186/2372] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:44 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:44.962631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e23229-095b-44ad-b8ca-fa7e1ac40488 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:44.972776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e23229-095b-44ad-b8ca-fa7e1ac40488 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:44.977273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4bb8f-5a30-4fdc-acf5-1e145a7d1e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:44.978111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4bb8f-5a30-4fdc-acf5-1e145a7d1e51 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:44.978386 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4bb8f-5a30-4fdc-acf5-1e145a7d1e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:44.978656 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4bb8f-5a30-4fdc-acf5-1e145a7d1e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:44.979289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4bb8f-5a30-4fdc-acf5-1e145a7d1e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:44.980904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4bb8f-5a30-4fdc-acf5-1e145a7d1e51 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:44.991396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1187/2373] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:44.995237 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91769953-ddbd-4536-9bfc-f82cd67084e7 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.006588 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91769953-ddbd-4536-9bfc-f82cd67084e7 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.011347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4e502-5655-4be5-968f-70b642683842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.011970 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4e502-5655-4be5-968f-70b642683842 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.012253 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4e502-5655-4be5-968f-70b642683842 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.012570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4e502-5655-4be5-968f-70b642683842 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.013211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4e502-5655-4be5-968f-70b642683842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:45.014808 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4e502-5655-4be5-968f-70b642683842 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:45.025900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1187/2374] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:45.029968 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4bb8f-5a30-4fdc-acf5-1e145a7d1e51 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.040094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4bb8f-5a30-4fdc-acf5-1e145a7d1e51 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.044717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642f3ab-25f2-4106-b107-fb06d9d16593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.045354 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f3ab-25f2-4106-b107-fb06d9d16593 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.045631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f3ab-25f2-4106-b107-fb06d9d16593 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.045901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f3ab-25f2-4106-b107-fb06d9d16593 admin admin] PATH_INFO: `/v3/projects/6b660560fd5c43b1b8f0f76732a71bd2/users/2cfc9991d8094ee5942397a3a84de29f/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.060398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642f3ab-25f2-4106-b107-fb06d9d16593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b660560fd5c43b1b8f0f76732a71bd2, user_id=2cfc9991d8094ee5942397a3a84de29f, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:45.063619 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642f3ab-25f2-4106-b107-fb06d9d16593 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:45.075532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1188/2375] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:45 2022] PUT /identity/v3/projects/6b660560fd5c43b1b8f0f76732a71bd2/users/2cfc9991d8094ee5942397a3a84de29f/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:45.086238 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382d9d7-b232-47eb-aebf-bc5d91e614d3 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.086462 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382d9d7-b232-47eb-aebf-bc5d91e614d3 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.086741 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382d9d7-b232-47eb-aebf-bc5d91e614d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.116077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9382d9d7-b232-47eb-aebf-bc5d91e614d3 None None] MFA Rules not processed for user `2cfc9991d8094ee5942397a3a84de29f`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:45.117338 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9382d9d7-b232-47eb-aebf-bc5d91e614d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:45.122265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:45.182414 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1188/2376] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:45.186886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642f3ab-25f2-4106-b107-fb06d9d16593 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.205921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:45.248202 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642f3ab-25f2-4106-b107-fb06d9d16593 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.253173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61319e69-db99-4e72-9456-29a8afdba8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.253760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61319e69-db99-4e72-9456-29a8afdba8ec admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.254036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61319e69-db99-4e72-9456-29a8afdba8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.254306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61319e69-db99-4e72-9456-29a8afdba8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.254924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61319e69-db99-4e72-9456-29a8afdba8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:45.256852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61319e69-db99-4e72-9456-29a8afdba8ec admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:45.282728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1189/2377] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:45 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:45.286953 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382d9d7-b232-47eb-aebf-bc5d91e614d3 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.297354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382d9d7-b232-47eb-aebf-bc5d91e614d3 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.373602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf325b-dc53-4d3e-b96b-3a2ff6bd23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.374219 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf325b-dc53-4d3e-b96b-3a2ff6bd23c1 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.374501 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf325b-dc53-4d3e-b96b-3a2ff6bd23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.374777 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf325b-dc53-4d3e-b96b-3a2ff6bd23c1 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.376098 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf325b-dc53-4d3e-b96b-3a2ff6bd23c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:45.378048 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf325b-dc53-4d3e-b96b-3a2ff6bd23c1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:45.405375 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1189/2378] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:45 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:45.406758 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61319e69-db99-4e72-9456-29a8afdba8ec admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.418846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61319e69-db99-4e72-9456-29a8afdba8ec admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.424913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4ad34-b0b6-4375-a394-c8f85c4b0d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.425538 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4ad34-b0b6-4375-a394-c8f85c4b0d70 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.425844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4ad34-b0b6-4375-a394-c8f85c4b0d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.426093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4ad34-b0b6-4375-a394-c8f85c4b0d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.427446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4ad34-b0b6-4375-a394-c8f85c4b0d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:45.429144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4ad34-b0b6-4375-a394-c8f85c4b0d70 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:45.454825 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1190/2379] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:45.457484 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf325b-dc53-4d3e-b96b-3a2ff6bd23c1 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.471720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf325b-dc53-4d3e-b96b-3a2ff6bd23c1 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.476968 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175ce0f-cb1e-4441-8547-d0eb45ce8b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.477913 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175ce0f-cb1e-4441-8547-d0eb45ce8b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.478199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175ce0f-cb1e-4441-8547-d0eb45ce8b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.478639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175ce0f-cb1e-4441-8547-d0eb45ce8b99 admin admin] PATH_INFO: `/v3/projects/cf4cc1c9d89b4e04a7bf8b383854c736/users/83e99f4093b34da4b83a26830d1b73db/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.498114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175ce0f-cb1e-4441-8547-d0eb45ce8b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4cc1c9d89b4e04a7bf8b383854c736, user_id=83e99f4093b34da4b83a26830d1b73db, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:45.501393 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175ce0f-cb1e-4441-8547-d0eb45ce8b99 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:45.518343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1190/2380] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:45 2022] PUT /identity/v3/projects/cf4cc1c9d89b4e04a7bf8b383854c736/users/83e99f4093b34da4b83a26830d1b73db/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:45.522287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4ad34-b0b6-4375-a394-c8f85c4b0d70 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.532904 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5ecee-5dec-43ae-a88d-67514cfabd97 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.533246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5ecee-5dec-43ae-a88d-67514cfabd97 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.533681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5ecee-5dec-43ae-a88d-67514cfabd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.535406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4ad34-b0b6-4375-a394-c8f85c4b0d70 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.555640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-94c5ecee-5dec-43ae-a88d-67514cfabd97 None None] MFA Rules not processed for user `36a281f7f9c8435c8ad65ae4b0bf141a`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:45.558448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-94c5ecee-5dec-43ae-a88d-67514cfabd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:45.638639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce351d-5c72-4ca9-92c6-39759fa78919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.639590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce351d-5c72-4ca9-92c6-39759fa78919 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.639975 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce351d-5c72-4ca9-92c6-39759fa78919 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.640423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce351d-5c72-4ca9-92c6-39759fa78919 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.642297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce351d-5c72-4ca9-92c6-39759fa78919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:45.643918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce351d-5c72-4ca9-92c6-39759fa78919 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:45.644521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1191/2381] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:45.654450 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1191/2382] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:45.659521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5ecee-5dec-43ae-a88d-67514cfabd97 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.675642 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5ecee-5dec-43ae-a88d-67514cfabd97 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.682325 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf33a84-1693-4c43-a14a-9b034ccd04a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.683079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf33a84-1693-4c43-a14a-9b034ccd04a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.683415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf33a84-1693-4c43-a14a-9b034ccd04a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.683670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf33a84-1693-4c43-a14a-9b034ccd04a0 admin admin] PATH_INFO: `/v3/projects/cf4cc1c9d89b4e04a7bf8b383854c736/users/83e99f4093b34da4b83a26830d1b73db/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.685636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf33a84-1693-4c43-a14a-9b034ccd04a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4cc1c9d89b4e04a7bf8b383854c736, user_id=83e99f4093b34da4b83a26830d1b73db, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:45.691730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf33a84-1693-4c43-a14a-9b034ccd04a0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:45.709092 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1192/2383] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:45 2022] PUT /identity/v3/projects/cf4cc1c9d89b4e04a7bf8b383854c736/users/83e99f4093b34da4b83a26830d1b73db/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:45.723503 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29595e4b-098b-45f1-8b34-c8b281a7488d None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.723617 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29595e4b-098b-45f1-8b34-c8b281a7488d None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.723847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29595e4b-098b-45f1-8b34-c8b281a7488d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.761063 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-29595e4b-098b-45f1-8b34-c8b281a7488d None None] MFA Rules not processed for user `83e99f4093b34da4b83a26830d1b73db`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:45.762363 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-29595e4b-098b-45f1-8b34-c8b281a7488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:45.841791 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf33a84-1693-4c43-a14a-9b034ccd04a0 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.843812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-bdf33a84-1693-4c43-a14a-9b034ccd04a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:45.854930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:45.892606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1192/2384] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:45.905167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29595e4b-098b-45f1-8b34-c8b281a7488d None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:45.909043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-29595e4b-098b-45f1-8b34-c8b281a7488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:45.926799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:45.934499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf33a84-1693-4c43-a14a-9b034ccd04a0 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.939622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad662f0-1f14-43e3-9a37-5c1fe5b81b34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.940576 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad662f0-1f14-43e3-9a37-5c1fe5b81b34 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.940806 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad662f0-1f14-43e3-9a37-5c1fe5b81b34 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.941179 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad662f0-1f14-43e3-9a37-5c1fe5b81b34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:45.943511 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-cad662f0-1f14-43e3-9a37-5c1fe5b81b34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:45.991028 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29595e4b-098b-45f1-8b34-c8b281a7488d None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:45.995369 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f3e8e-8ecf-4d03-b6e1-7e233d669e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:45.996555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f3e8e-8ecf-4d03-b6e1-7e233d669e9f service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:45.996835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f3e8e-8ecf-4d03-b6e1-7e233d669e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:45.997114 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f3e8e-8ecf-4d03-b6e1-7e233d669e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:46.010266 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f3e8e-8ecf-4d03-b6e1-7e233d669e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:46.012396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f3e8e-8ecf-4d03-b6e1-7e233d669e9f service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:46.025726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1193/2385] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:46.036500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad662f0-1f14-43e3-9a37-5c1fe5b81b34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:46.038566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad662f0-1f14-43e3-9a37-5c1fe5b81b34 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:46.050447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1193/2386] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:46.625756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f3e8e-8ecf-4d03-b6e1-7e233d669e9f service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:46.730836 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f3e8e-8ecf-4d03-b6e1-7e233d669e9f service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:46.737636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:46.738377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:46.738711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:46.739037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] PATH_INFO: `/v3/users/2cfc9991d8094ee5942397a3a84de29f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:46.740548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cfc9991d8094ee5942397a3a84de29f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:46.743229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:46.838216 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cfc9991d8094ee5942397a3a84de29f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:46.847274 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cfc9991d8094ee5942397a3a84de29f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:46.853277 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cfc9991d8094ee5942397a3a84de29f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:46.854134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cfc9991d8094ee5942397a3a84de29f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:46.881316 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cfc9991d8094ee5942397a3a84de29f'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:46.916128 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1194/2387] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:46 2022] DELETE /identity/v3/users/2cfc9991d8094ee5942397a3a84de29f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:46.921342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25a45d-e402-4d12-830c-167821a90b09 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:46.921867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25a45d-e402-4d12-830c-167821a90b09 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:46.922196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25a45d-e402-4d12-830c-167821a90b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:46.958164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-7d25a45d-e402-4d12-830c-167821a90b09 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:46.958891 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7d25a45d-e402-4d12-830c-167821a90b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:46.983286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:46.986012 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:47.031820 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:47.048418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1194/2388] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:47.055711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25a45d-e402-4d12-830c-167821a90b09 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:47.057377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-7d25a45d-e402-4d12-830c-167821a90b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:47.058920 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e05eb-bdde-433b-98d2-8582f2a5967d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:47.063703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:47.064781 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:47.065271 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:47.066120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:47.068522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:47.125893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25a45d-e402-4d12-830c-167821a90b09 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:47.129471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed5a6d-8097-47dd-82da-2a9121efcb84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:47.130069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed5a6d-8097-47dd-82da-2a9121efcb84 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:47.130345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed5a6d-8097-47dd-82da-2a9121efcb84 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:47.130615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed5a6d-8097-47dd-82da-2a9121efcb84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:47.137471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:47.138864 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:47.139597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed5a6d-8097-47dd-82da-2a9121efcb84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:47.141297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed5a6d-8097-47dd-82da-2a9121efcb84 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:47.148467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1195/2389] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:48:47.151363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1195/2390] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:47.433822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:47.435329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:47.449326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4517d-7aa1-4448-9f56-c53c928d0f59 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:47.455362 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:47.455998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:47.456279 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:47.456625 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] PATH_INFO: `/v3/projects/6b660560fd5c43b1b8f0f76732a71bd2` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:47.457713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b660560fd5c43b1b8f0f76732a71bd2)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:47.459640 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:47.507608 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b660560fd5c43b1b8f0f76732a71bd2', 'request_id': 'req-52406892-5946-4563-94cd-4f7ff2a6396d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.513461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b660560fd5c43b1b8f0f76732a71bd2', 'request_id': 'req-52406892-5946-4563-94cd-4f7ff2a6396d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.522077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b660560fd5c43b1b8f0f76732a71bd2', 'request_id': 'req-52406892-5946-4563-94cd-4f7ff2a6396d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.540912 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] The token cache is being invalidate because project 6b660560fd5c43b1b8f0f76732a71bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:47.541233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.541984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.544168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1196/2391] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:47 2022] DELETE /identity/v3/projects/6b660560fd5c43b1b8f0f76732a71bd2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:47.549461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed5a6d-8097-47dd-82da-2a9121efcb84 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:47.554553 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:47.560245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed5a6d-8097-47dd-82da-2a9121efcb84 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:47.629234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:47.629876 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:47.630154 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:47.630857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] PATH_INFO: `/v3/users/83e99f4093b34da4b83a26830d1b73db` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:47.631681 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e99f4093b34da4b83a26830d1b73db)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:47.633733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:47.680534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e99f4093b34da4b83a26830d1b73db'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.687150 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e99f4093b34da4b83a26830d1b73db'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.688288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e99f4093b34da4b83a26830d1b73db'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.688999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e99f4093b34da4b83a26830d1b73db'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.695985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c5099a27-d02b-488a-bf95-b48b7857a61a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e99f4093b34da4b83a26830d1b73db'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:47.713755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1196/2392] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:47 2022] DELETE /identity/v3/users/83e99f4093b34da4b83a26830d1b73db => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:47.999146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:47.999146 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:48.014289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52406892-5946-4563-94cd-4f7ff2a6396d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:48.107675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:48.108566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.108984 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.110086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] PATH_INFO: `/v3/projects/cf4cc1c9d89b4e04a7bf8b383854c736` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.112029 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf4cc1c9d89b4e04a7bf8b383854c736)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:48.114871 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:48.130724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:48.188584 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf4cc1c9d89b4e04a7bf8b383854c736', 'request_id': 'req-aa472d78-bee8-4cf1-ba09-870fd521d347'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:48.214401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf4cc1c9d89b4e04a7bf8b383854c736', 'request_id': 'req-aa472d78-bee8-4cf1-ba09-870fd521d347'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:48.214401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf4cc1c9d89b4e04a7bf8b383854c736', 'request_id': 'req-aa472d78-bee8-4cf1-ba09-870fd521d347'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:48.218827 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] The token cache is being invalidate because project cf4cc1c9d89b4e04a7bf8b383854c736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:48.219121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:48.219879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-aa472d78-bee8-4cf1-ba09-870fd521d347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:48.221603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1197/2393] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:47 2022] DELETE /identity/v3/projects/cf4cc1c9d89b4e04a7bf8b383854c736 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:48.226548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b4567-0bc3-46e2-b148-9aac9ceed0c5 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.227110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b4567-0bc3-46e2-b148-9aac9ceed0c5 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.227252 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b4567-0bc3-46e2-b148-9aac9ceed0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.255561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-fb9b4567-0bc3-46e2-b148-9aac9ceed0c5 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:48.257158 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fb9b4567-0bc3-46e2-b148-9aac9ceed0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:48.326299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1197/2394] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:48.335170 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed9ab-23b7-4a97-814d-620740405cb8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.335465 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed9ab-23b7-4a97-814d-620740405cb8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.335750 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed9ab-23b7-4a97-814d-620740405cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.361341 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-718ed9ab-23b7-4a97-814d-620740405cb8 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:48.362533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-718ed9ab-23b7-4a97-814d-620740405cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:48.370655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1198/2395] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:48.375192 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b4567-0bc3-46e2-b148-9aac9ceed0c5 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:48.384834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b4567-0bc3-46e2-b148-9aac9ceed0c5 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:48.389453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679ea9b-bef0-47b9-ba08-877f443306b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:48.390073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679ea9b-bef0-47b9-ba08-877f443306b8 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.390346 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679ea9b-bef0-47b9-ba08-877f443306b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.390661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679ea9b-bef0-47b9-ba08-877f443306b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.391305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679ea9b-bef0-47b9-ba08-877f443306b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:48.391597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679ea9b-bef0-47b9-ba08-877f443306b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:48.393216 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679ea9b-bef0-47b9-ba08-877f443306b8 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:48.409372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1198/2396] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:48:48.410534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed9ab-23b7-4a97-814d-620740405cb8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:48.419805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed9ab-23b7-4a97-814d-620740405cb8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:48.425992 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1d098-3f10-4d14-8d6a-3c7ba487bc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:48.426657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1d098-3f10-4d14-8d6a-3c7ba487bc4d admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.426947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1d098-3f10-4d14-8d6a-3c7ba487bc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.427211 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1d098-3f10-4d14-8d6a-3c7ba487bc4d admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.427844 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1d098-3f10-4d14-8d6a-3c7ba487bc4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:48.429773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1d098-3f10-4d14-8d6a-3c7ba487bc4d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:48.458946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1199/2397] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:48 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:48.463816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679ea9b-bef0-47b9-ba08-877f443306b8 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:48.479249 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679ea9b-bef0-47b9-ba08-877f443306b8 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:48.484539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:48.601643 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c87a8c-e8a8-493d-8cd9-b745847032a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:48.602269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c87a8c-e8a8-493d-8cd9-b745847032a3 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.602882 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c87a8c-e8a8-493d-8cd9-b745847032a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.605907 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c87a8c-e8a8-493d-8cd9-b745847032a3 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.606721 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c87a8c-e8a8-493d-8cd9-b745847032a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:48.609273 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c87a8c-e8a8-493d-8cd9-b745847032a3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:48.641585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1199/2398] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:48 2022] POST /identity/v3/users => generated 279 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:48.643143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1d098-3f10-4d14-8d6a-3c7ba487bc4d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:48.665282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1d098-3f10-4d14-8d6a-3c7ba487bc4d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:48.672766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51c6d7-8297-49c7-a44e-d0e83a3c338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:48.673580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51c6d7-8297-49c7-a44e-d0e83a3c338d admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.674777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51c6d7-8297-49c7-a44e-d0e83a3c338d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.675144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51c6d7-8297-49c7-a44e-d0e83a3c338d admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.675942 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51c6d7-8297-49c7-a44e-d0e83a3c338d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:48.678009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51c6d7-8297-49c7-a44e-d0e83a3c338d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:48.694420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1200/2399] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:48.696101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c87a8c-e8a8-493d-8cd9-b745847032a3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:48.710006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c87a8c-e8a8-493d-8cd9-b745847032a3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:48.718846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a4a9d-8ccf-494f-83d8-72253174079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:48.719694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a4a9d-8ccf-494f-83d8-72253174079a admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.719978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a4a9d-8ccf-494f-83d8-72253174079a admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.720254 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a4a9d-8ccf-494f-83d8-72253174079a admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.721018 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a4a9d-8ccf-494f-83d8-72253174079a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:48.723332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a4a9d-8ccf-494f-83d8-72253174079a admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:48.750910 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1200/2400] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:48.754811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51c6d7-8297-49c7-a44e-d0e83a3c338d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:48.765398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51c6d7-8297-49c7-a44e-d0e83a3c338d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:48.779773 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036f02-be2b-4790-afc9-15bb0625f994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:48.782398 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036f02-be2b-4790-afc9-15bb0625f994 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.782796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036f02-be2b-4790-afc9-15bb0625f994 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.783145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036f02-be2b-4790-afc9-15bb0625f994 admin admin] PATH_INFO: `/v3/projects/2bb4f76c9f0247eeac4cc180dd5a11c0/users/c08c28e36d5a4f05847e2940efdf6b7c/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.806885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af036f02-be2b-4790-afc9-15bb0625f994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb4f76c9f0247eeac4cc180dd5a11c0, user_id=c08c28e36d5a4f05847e2940efdf6b7c, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:48.809665 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af036f02-be2b-4790-afc9-15bb0625f994 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:48.823506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1201/2401] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:48 2022] PUT /identity/v3/projects/2bb4f76c9f0247eeac4cc180dd5a11c0/users/c08c28e36d5a4f05847e2940efdf6b7c/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:48.834130 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e694f-a3ab-45bd-9445-51f6bcdafd98 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:48.834444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e694f-a3ab-45bd-9445-51f6bcdafd98 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:48.834718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e694f-a3ab-45bd-9445-51f6bcdafd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:48.864855 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-494e694f-a3ab-45bd-9445-51f6bcdafd98 None None] MFA Rules not processed for user `c08c28e36d5a4f05847e2940efdf6b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:48.866220 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-494e694f-a3ab-45bd-9445-51f6bcdafd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:48.939344 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1201/2402] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:48.944489 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036f02-be2b-4790-afc9-15bb0625f994 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:49.007694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036f02-be2b-4790-afc9-15bb0625f994 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:49.012732 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003abba4-b464-4276-9f22-133cb6db1f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:49.013344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003abba4-b464-4276-9f22-133cb6db1f6b admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:49.013622 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003abba4-b464-4276-9f22-133cb6db1f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:49.013897 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003abba4-b464-4276-9f22-133cb6db1f6b admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:49.014528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003abba4-b464-4276-9f22-133cb6db1f6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:49.016905 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003abba4-b464-4276-9f22-133cb6db1f6b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:49.043954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1202/2403] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:48 2022] POST /identity/v3/projects => generated 355 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:49.048853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e694f-a3ab-45bd-9445-51f6bcdafd98 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:49.059965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e694f-a3ab-45bd-9445-51f6bcdafd98 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:49.121941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1987e-49c1-46b1-9db4-83c69c54fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:49.122569 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1987e-49c1-46b1-9db4-83c69c54fc24 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:49.122886 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1987e-49c1-46b1-9db4-83c69c54fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:49.123194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1987e-49c1-46b1-9db4-83c69c54fc24 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:49.123858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1987e-49c1-46b1-9db4-83c69c54fc24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:49.126465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1987e-49c1-46b1-9db4-83c69c54fc24 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:49.148993 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1202/2404] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:49 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:49.153060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003abba4-b464-4276-9f22-133cb6db1f6b admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:49.163583 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003abba4-b464-4276-9f22-133cb6db1f6b admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:49.168317 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a0733-cbb1-4c9a-b51d-8c05ad3406ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:49.168943 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a0733-cbb1-4c9a-b51d-8c05ad3406ba admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:49.169228 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a0733-cbb1-4c9a-b51d-8c05ad3406ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:49.169505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a0733-cbb1-4c9a-b51d-8c05ad3406ba admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:49.170147 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a0733-cbb1-4c9a-b51d-8c05ad3406ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:49.171777 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a0733-cbb1-4c9a-b51d-8c05ad3406ba admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:49.182536 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1203/2405] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:49.186636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1987e-49c1-46b1-9db4-83c69c54fc24 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:49.197377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1987e-49c1-46b1-9db4-83c69c54fc24 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:49.202015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d728a-e19f-41b5-9e25-aecf22bb5c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:49.202690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d728a-e19f-41b5-9e25-aecf22bb5c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:49.203007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d728a-e19f-41b5-9e25-aecf22bb5c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:49.203465 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d728a-e19f-41b5-9e25-aecf22bb5c7c admin admin] PATH_INFO: `/v3/projects/b7d216ac0c2247d39fd4970dde945fe5/users/45839c45fa0f4aa0874f044af1abaab2/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:49.205730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:49.221482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d728a-e19f-41b5-9e25-aecf22bb5c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d216ac0c2247d39fd4970dde945fe5, user_id=45839c45fa0f4aa0874f044af1abaab2, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:49.224548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d728a-e19f-41b5-9e25-aecf22bb5c7c admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:49.241827 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1203/2406] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:49 2022] PUT /identity/v3/projects/b7d216ac0c2247d39fd4970dde945fe5/users/45839c45fa0f4aa0874f044af1abaab2/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:49.245778 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a0733-cbb1-4c9a-b51d-8c05ad3406ba admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:49.257233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a0733-cbb1-4c9a-b51d-8c05ad3406ba admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:49.282498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:49.358718 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5471d3-df4c-425f-891d-dea2710bbe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:49.358718 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5471d3-df4c-425f-891d-dea2710bbe97 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:49.359179 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5471d3-df4c-425f-891d-dea2710bbe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:49.359715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5471d3-df4c-425f-891d-dea2710bbe97 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:49.360522 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5471d3-df4c-425f-891d-dea2710bbe97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:49.362350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5471d3-df4c-425f-891d-dea2710bbe97 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:49.382478 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1204/2407] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:49.386691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d728a-e19f-41b5-9e25-aecf22bb5c7c admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:49.398969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d728a-e19f-41b5-9e25-aecf22bb5c7c admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:49.403961 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47165222-c1d0-43f6-8fb7-e9484ec539a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:49.404678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47165222-c1d0-43f6-8fb7-e9484ec539a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:49.404999 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47165222-c1d0-43f6-8fb7-e9484ec539a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:49.405385 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47165222-c1d0-43f6-8fb7-e9484ec539a9 admin admin] PATH_INFO: `/v3/projects/b7d216ac0c2247d39fd4970dde945fe5/users/45839c45fa0f4aa0874f044af1abaab2/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:49.408089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47165222-c1d0-43f6-8fb7-e9484ec539a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d216ac0c2247d39fd4970dde945fe5, user_id=45839c45fa0f4aa0874f044af1abaab2, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:49.410744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47165222-c1d0-43f6-8fb7-e9484ec539a9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:49.421445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1204/2408] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:49 2022] PUT /identity/v3/projects/b7d216ac0c2247d39fd4970dde945fe5/users/45839c45fa0f4aa0874f044af1abaab2/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:49.433227 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0257d-3412-4e7d-b924-b07013e093d8 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:49.433520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0257d-3412-4e7d-b924-b07013e093d8 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:49.433799 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0257d-3412-4e7d-b924-b07013e093d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:49.465738 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-cfa0257d-3412-4e7d-b924-b07013e093d8 None None] MFA Rules not processed for user `45839c45fa0f4aa0874f044af1abaab2`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:49.466576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cfa0257d-3412-4e7d-b924-b07013e093d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:49.548249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1205/2409] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:49.559460 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47165222-c1d0-43f6-8fb7-e9484ec539a9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:49.562924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-47165222-c1d0-43f6-8fb7-e9484ec539a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:49.669038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47165222-c1d0-43f6-8fb7-e9484ec539a9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:49.683552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d943a2-bfaf-491d-a5db-c94c8304d868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:49.684691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d943a2-bfaf-491d-a5db-c94c8304d868 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:49.685231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d943a2-bfaf-491d-a5db-c94c8304d868 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:49.685694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d943a2-bfaf-491d-a5db-c94c8304d868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:49.698932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d943a2-bfaf-491d-a5db-c94c8304d868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:49.700566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d943a2-bfaf-491d-a5db-c94c8304d868 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:49.725392 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1205/2410] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:49.846098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0257d-3412-4e7d-b924-b07013e093d8 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:49.864542 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0257d-3412-4e7d-b924-b07013e093d8 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:49.879890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069cef8-1d76-4887-9e14-6fc69ccf1654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:49.880562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069cef8-1d76-4887-9e14-6fc69ccf1654 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:49.880845 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069cef8-1d76-4887-9e14-6fc69ccf1654 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:49.881208 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069cef8-1d76-4887-9e14-6fc69ccf1654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:49.983462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069cef8-1d76-4887-9e14-6fc69ccf1654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:49.986922 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069cef8-1d76-4887-9e14-6fc69ccf1654 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:50.002692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1206/2411] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:50.843107 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d943a2-bfaf-491d-a5db-c94c8304d868 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:50.845171 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-36d943a2-bfaf-491d-a5db-c94c8304d868 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:50.901489 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d943a2-bfaf-491d-a5db-c94c8304d868 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:50.904937 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38152f-98de-499f-8087-48c188c19a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:50.905525 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38152f-98de-499f-8087-48c188c19a28 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:50.905801 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38152f-98de-499f-8087-48c188c19a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:50.906072 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38152f-98de-499f-8087-48c188c19a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:50.908456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-fc38152f-98de-499f-8087-48c188c19a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:50.970209 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38152f-98de-499f-8087-48c188c19a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:50.971710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38152f-98de-499f-8087-48c188c19a28 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:50.981347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1206/2412] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:51.313409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069cef8-1d76-4887-9e14-6fc69ccf1654 service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:51.314963 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-9069cef8-1d76-4887-9e14-6fc69ccf1654 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:51.388076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:51.444805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069cef8-1d76-4887-9e14-6fc69ccf1654 service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:51.450303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:51.451135 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:51.451482 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:51.451769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] PATH_INFO: `/v3/users/e11837cf9f73454eb81b5c1de6192f3c` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:51.453370 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11837cf9f73454eb81b5c1de6192f3c)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:51.455834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:51.544734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11837cf9f73454eb81b5c1de6192f3c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:51.552829 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11837cf9f73454eb81b5c1de6192f3c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:51.553862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11837cf9f73454eb81b5c1de6192f3c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:51.554630 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11837cf9f73454eb81b5c1de6192f3c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:51.564181 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11837cf9f73454eb81b5c1de6192f3c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:51.584769 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1207/2413] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:51 2022] DELETE /identity/v3/users/e11837cf9f73454eb81b5c1de6192f3c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:51.589279 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a033ae-8369-4d6a-976f-1ce78c2b037e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:51.589570 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a033ae-8369-4d6a-976f-1ce78c2b037e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:51.589851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a033ae-8369-4d6a-976f-1ce78c2b037e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:51.609583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-02a033ae-8369-4d6a-976f-1ce78c2b037e None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:51.610915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-02a033ae-8369-4d6a-976f-1ce78c2b037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:51.628408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:51.682646 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1207/2414] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:51.693136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:51.694636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:51.754764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf46f02-b6bf-47b1-9671-d30a13c81715 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:51.758618 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596820d0-142c-4e5f-82d7-a4361d11862e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:51.759382 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596820d0-142c-4e5f-82d7-a4361d11862e service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:51.759679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596820d0-142c-4e5f-82d7-a4361d11862e service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:51.760804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596820d0-142c-4e5f-82d7-a4361d11862e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:51.770091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596820d0-142c-4e5f-82d7-a4361d11862e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:51.771528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596820d0-142c-4e5f-82d7-a4361d11862e service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:51.781152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1208/2415] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:52.060374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a033ae-8369-4d6a-976f-1ce78c2b037e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:52.062558 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-02a033ae-8369-4d6a-976f-1ce78c2b037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:52.075520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a033ae-8369-4d6a-976f-1ce78c2b037e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:52.081804 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:52.082641 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:52.083007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:52.083278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] PATH_INFO: `/v3/projects/e805da7ed5a84d469a2a8c3f5bbc95aa` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:52.084790 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e805da7ed5a84d469a2a8c3f5bbc95aa)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:52.086730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:52.137517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e805da7ed5a84d469a2a8c3f5bbc95aa', 'request_id': 'req-aa9116ed-f761-40f7-8cf4-9bd52a236465'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.143348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e805da7ed5a84d469a2a8c3f5bbc95aa', 'request_id': 'req-aa9116ed-f761-40f7-8cf4-9bd52a236465'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.153660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e805da7ed5a84d469a2a8c3f5bbc95aa', 'request_id': 'req-aa9116ed-f761-40f7-8cf4-9bd52a236465'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.166799 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] The token cache is being invalidate because project e805da7ed5a84d469a2a8c3f5bbc95aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:52.167227 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.167892 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.169445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1208/2416] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:52 2022] DELETE /identity/v3/projects/e805da7ed5a84d469a2a8c3f5bbc95aa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:52.174370 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596820d0-142c-4e5f-82d7-a4361d11862e service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:52.184019 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596820d0-142c-4e5f-82d7-a4361d11862e service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:52.236293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:52.247741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:52.248673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:52.249011 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:52.249570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] PATH_INFO: `/v3/users/49d7de7930ba44b8ad663c13dfdd0782` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:52.251727 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=49d7de7930ba44b8ad663c13dfdd0782)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:52.253801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:52.303305 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49d7de7930ba44b8ad663c13dfdd0782'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.310569 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49d7de7930ba44b8ad663c13dfdd0782'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.311254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49d7de7930ba44b8ad663c13dfdd0782'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.311861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49d7de7930ba44b8ad663c13dfdd0782'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.321027 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49d7de7930ba44b8ad663c13dfdd0782'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:52.342563 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1209/2417] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:52 2022] DELETE /identity/v3/users/49d7de7930ba44b8ad663c13dfdd0782 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:52.869568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:52.871403 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:52.903204 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9116ed-f761-40f7-8cf4-9bd52a236465 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:52.990596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:52.991561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:52.991959 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:52.992352 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] PATH_INFO: `/v3/projects/a8a726ebbf624cea85440e642d0b6ec6` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:52.993668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a726ebbf624cea85440e642d0b6ec6)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:52.996871 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:53.093426 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:53.099673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a726ebbf624cea85440e642d0b6ec6', 'request_id': 'req-4cf020a4-e2de-4210-b219-0a04b67bdcbd'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.114640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a726ebbf624cea85440e642d0b6ec6', 'request_id': 'req-4cf020a4-e2de-4210-b219-0a04b67bdcbd'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.128544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a726ebbf624cea85440e642d0b6ec6', 'request_id': 'req-4cf020a4-e2de-4210-b219-0a04b67bdcbd'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.163360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] The token cache is being invalidate because project a8a726ebbf624cea85440e642d0b6ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:53.163858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.164761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.168255 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1209/2418] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:52 2022] DELETE /identity/v3/projects/a8a726ebbf624cea85440e642d0b6ec6 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:53.172297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:53.174539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:53.189343 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dd831-f37b-4772-b695-1673afc617ac admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:53.286837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:53.287780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:53.288068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:53.288443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] PATH_INFO: `/v3/users/46f694bf8e044c4a8979301762348fcf` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:53.289694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f694bf8e044c4a8979301762348fcf)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:53.291991 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:53.343371 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f694bf8e044c4a8979301762348fcf'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.352681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f694bf8e044c4a8979301762348fcf'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.353400 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f694bf8e044c4a8979301762348fcf'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.354378 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f694bf8e044c4a8979301762348fcf'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.362111 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-7eba83d5-f4c5-418e-8e53-8ea1d7f77660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f694bf8e044c4a8979301762348fcf'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.369299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:53.385796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1210/2419] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:53 2022] DELETE /identity/v3/users/46f694bf8e044c4a8979301762348fcf => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:53.670270 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:53.672720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:53.743199 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf020a4-e2de-4210-b219-0a04b67bdcbd admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:53.827438 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:53.829137 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:53.829711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:53.830147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] PATH_INFO: `/v3/projects/4d73256f1b7747dba53a039c729c2998` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:53.831485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d73256f1b7747dba53a039c729c2998)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:53.844286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:53.908119 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d73256f1b7747dba53a039c729c2998', 'request_id': 'req-514fbddd-8105-4236-bdbd-56ae5b29367b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.915554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d73256f1b7747dba53a039c729c2998', 'request_id': 'req-514fbddd-8105-4236-bdbd-56ae5b29367b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.923654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d73256f1b7747dba53a039c729c2998', 'request_id': 'req-514fbddd-8105-4236-bdbd-56ae5b29367b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.936709 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] The token cache is being invalidate because project 4d73256f1b7747dba53a039c729c2998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:48:53.937009 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.938218 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-514fbddd-8105-4236-bdbd-56ae5b29367b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:53.939834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1210/2420] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:48:53 2022] DELETE /identity/v3/projects/4d73256f1b7747dba53a039c729c2998 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:53.944875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26a89d-cc36-4ca1-b72a-3994083ab535 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:53.945159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26a89d-cc36-4ca1-b72a-3994083ab535 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:53.945436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26a89d-cc36-4ca1-b72a-3994083ab535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.018221 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-fa26a89d-cc36-4ca1-b72a-3994083ab535 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:54.019427 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-fa26a89d-cc36-4ca1-b72a-3994083ab535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:54.103962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1211/2421] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:54.112541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2799ed-e8e2-4932-98ac-4515501989e4 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.113022 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2799ed-e8e2-4932-98ac-4515501989e4 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.113120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2799ed-e8e2-4932-98ac-4515501989e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.129453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:54.143011 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1d2799ed-e8e2-4932-98ac-4515501989e4 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:54.144262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1d2799ed-e8e2-4932-98ac-4515501989e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:54.152294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1211/2422] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:54.156700 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26a89d-cc36-4ca1-b72a-3994083ab535 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.172869 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26a89d-cc36-4ca1-b72a-3994083ab535 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.178533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a0e89-fb10-4b08-992a-54e9b6ad273e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.179423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a0e89-fb10-4b08-992a-54e9b6ad273e admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.179915 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a0e89-fb10-4b08-992a-54e9b6ad273e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.180433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a0e89-fb10-4b08-992a-54e9b6ad273e admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.181297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a0e89-fb10-4b08-992a-54e9b6ad273e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:48:54.181797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a0e89-fb10-4b08-992a-54e9b6ad273e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.183606 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a0e89-fb10-4b08-992a-54e9b6ad273e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.200893 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1212/2423] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:48:54.202026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2799ed-e8e2-4932-98ac-4515501989e4 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.213510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2799ed-e8e2-4932-98ac-4515501989e4 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.219532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0febb9-c20c-46f9-91cb-1635ab510a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.220509 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0febb9-c20c-46f9-91cb-1635ab510a64 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.220990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0febb9-c20c-46f9-91cb-1635ab510a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.221554 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0febb9-c20c-46f9-91cb-1635ab510a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.222749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0febb9-c20c-46f9-91cb-1635ab510a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.225246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0febb9-c20c-46f9-91cb-1635ab510a64 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.261939 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1212/2424] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:54 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:54.267267 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a0e89-fb10-4b08-992a-54e9b6ad273e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.277765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a0e89-fb10-4b08-992a-54e9b6ad273e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.348460 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db7ebd-f09c-4118-b8a5-6d510d033ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.349794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db7ebd-f09c-4118-b8a5-6d510d033ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.350220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db7ebd-f09c-4118-b8a5-6d510d033ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.350655 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db7ebd-f09c-4118-b8a5-6d510d033ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.351438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db7ebd-f09c-4118-b8a5-6d510d033ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.353955 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db7ebd-f09c-4118-b8a5-6d510d033ce7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.391843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1213/2425] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:54 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:54.396149 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0febb9-c20c-46f9-91cb-1635ab510a64 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.408301 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0febb9-c20c-46f9-91cb-1635ab510a64 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.413639 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697abd9-d7d0-4f2f-8d58-bbe07c44b8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.414349 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697abd9-d7d0-4f2f-8d58-bbe07c44b8cf admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.414630 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697abd9-d7d0-4f2f-8d58-bbe07c44b8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.414902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697abd9-d7d0-4f2f-8d58-bbe07c44b8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.415543 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697abd9-d7d0-4f2f-8d58-bbe07c44b8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.417176 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697abd9-d7d0-4f2f-8d58-bbe07c44b8cf admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.429267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1213/2426] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:54.433415 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db7ebd-f09c-4118-b8a5-6d510d033ce7 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.445805 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db7ebd-f09c-4118-b8a5-6d510d033ce7 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.451249 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3391a4-6487-4d44-8984-24478f4d611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.452132 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3391a4-6487-4d44-8984-24478f4d611f admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.452545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3391a4-6487-4d44-8984-24478f4d611f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.453852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3391a4-6487-4d44-8984-24478f4d611f admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.454679 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3391a4-6487-4d44-8984-24478f4d611f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.456610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3391a4-6487-4d44-8984-24478f4d611f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.470634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1214/2427] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:54.476164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697abd9-d7d0-4f2f-8d58-bbe07c44b8cf admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.487483 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697abd9-d7d0-4f2f-8d58-bbe07c44b8cf admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.492812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b2e4b-18be-4f5d-8bab-8e97e8cec7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.493609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b2e4b-18be-4f5d-8bab-8e97e8cec7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.493993 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b2e4b-18be-4f5d-8bab-8e97e8cec7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.494515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b2e4b-18be-4f5d-8bab-8e97e8cec7cf admin admin] PATH_INFO: `/v3/projects/4099c8c7a4e44711832de4e19e2f8b81/users/f240bb91a0f84ba6a742788f77f79082/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.516816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b2e4b-18be-4f5d-8bab-8e97e8cec7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4099c8c7a4e44711832de4e19e2f8b81, user_id=f240bb91a0f84ba6a742788f77f79082, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.516816 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b2e4b-18be-4f5d-8bab-8e97e8cec7cf admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.535068 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1214/2428] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:54 2022] PUT /identity/v3/projects/4099c8c7a4e44711832de4e19e2f8b81/users/f240bb91a0f84ba6a742788f77f79082/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:54.547945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2544fdf-13e2-4a83-9aff-29fa4eb6adbf None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.547945 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2544fdf-13e2-4a83-9aff-29fa4eb6adbf None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.548206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2544fdf-13e2-4a83-9aff-29fa4eb6adbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.577930 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-f2544fdf-13e2-4a83-9aff-29fa4eb6adbf None None] MFA Rules not processed for user `f240bb91a0f84ba6a742788f77f79082`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:54.579432 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f2544fdf-13e2-4a83-9aff-29fa4eb6adbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:54.596879 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:54.662454 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1215/2429] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:54.667014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b2e4b-18be-4f5d-8bab-8e97e8cec7cf admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.739894 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b2e4b-18be-4f5d-8bab-8e97e8cec7cf admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.745682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448b93a-4cf2-4e7a-9f29-1342057b0384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.746298 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448b93a-4cf2-4e7a-9f29-1342057b0384 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.746580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448b93a-4cf2-4e7a-9f29-1342057b0384 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.747004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448b93a-4cf2-4e7a-9f29-1342057b0384 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.750350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448b93a-4cf2-4e7a-9f29-1342057b0384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.750350 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448b93a-4cf2-4e7a-9f29-1342057b0384 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.784394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1215/2430] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:54 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:54.786914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2544fdf-13e2-4a83-9aff-29fa4eb6adbf None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.801348 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2544fdf-13e2-4a83-9aff-29fa4eb6adbf None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.864895 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef6568-b754-41ff-b230-5df4bdf2f8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.865519 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef6568-b754-41ff-b230-5df4bdf2f8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.865798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef6568-b754-41ff-b230-5df4bdf2f8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.866072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef6568-b754-41ff-b230-5df4bdf2f8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.866714 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef6568-b754-41ff-b230-5df4bdf2f8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.868847 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef6568-b754-41ff-b230-5df4bdf2f8a0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.892965 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1216/2431] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:54 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:48:54.897138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448b93a-4cf2-4e7a-9f29-1342057b0384 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.908888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448b93a-4cf2-4e7a-9f29-1342057b0384 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.913995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d219fd1-ef10-40f4-b04e-9b593512e02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.914704 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d219fd1-ef10-40f4-b04e-9b593512e02d admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.914980 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d219fd1-ef10-40f4-b04e-9b593512e02d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.915251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d219fd1-ef10-40f4-b04e-9b593512e02d admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.915978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d219fd1-ef10-40f4-b04e-9b593512e02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.917649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d219fd1-ef10-40f4-b04e-9b593512e02d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.928930 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1216/2432] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:54.933186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef6568-b754-41ff-b230-5df4bdf2f8a0 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.943162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef6568-b754-41ff-b230-5df4bdf2f8a0 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.948667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45580c70-cbe5-43cd-aff9-08d778aafeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.949312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45580c70-cbe5-43cd-aff9-08d778aafeee admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.949633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45580c70-cbe5-43cd-aff9-08d778aafeee admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.949853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45580c70-cbe5-43cd-aff9-08d778aafeee admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:54.950502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45580c70-cbe5-43cd-aff9-08d778aafeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:54.952230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45580c70-cbe5-43cd-aff9-08d778aafeee admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:54.963882 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1217/2433] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:54.968262 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d219fd1-ef10-40f4-b04e-9b593512e02d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:54.980832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d219fd1-ef10-40f4-b04e-9b593512e02d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:54.985960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5be6f4-ef40-49e2-b6dd-c10cf2f770bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:54.986621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5be6f4-ef40-49e2-b6dd-c10cf2f770bb admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:54.988290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5be6f4-ef40-49e2-b6dd-c10cf2f770bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:54.988290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5be6f4-ef40-49e2-b6dd-c10cf2f770bb admin admin] PATH_INFO: `/v3/projects/bcfa46ad41fc4ce1a6d1da8b1352499b/users/312262ba49ec4679b2868a05cdb517ed/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.033932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5be6f4-ef40-49e2-b6dd-c10cf2f770bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcfa46ad41fc4ce1a6d1da8b1352499b, user_id=312262ba49ec4679b2868a05cdb517ed, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:55.037188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5be6f4-ef40-49e2-b6dd-c10cf2f770bb admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:55.052423 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1217/2434] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:54 2022] PUT /identity/v3/projects/bcfa46ad41fc4ce1a6d1da8b1352499b/users/312262ba49ec4679b2868a05cdb517ed/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:55.066934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a1452-94ea-4a7a-a07b-7e44e6d3a315 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.067766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a1452-94ea-4a7a-a07b-7e44e6d3a315 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.067766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a1452-94ea-4a7a-a07b-7e44e6d3a315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.104793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d36a1452-94ea-4a7a-a07b-7e44e6d3a315 None None] MFA Rules not processed for user `312262ba49ec4679b2868a05cdb517ed`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:55.106601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d36a1452-94ea-4a7a-a07b-7e44e6d3a315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:55.188695 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1218/2435] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:55.193297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5be6f4-ef40-49e2-b6dd-c10cf2f770bb admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:55.220814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442fbc3-6d18-484d-975c-f654c7ca2ac5 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.221555 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442fbc3-6d18-484d-975c-f654c7ca2ac5 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.221841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442fbc3-6d18-484d-975c-f654c7ca2ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.253854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:55.262550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5442fbc3-6d18-484d-975c-f654c7ca2ac5 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:55.263809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5442fbc3-6d18-484d-975c-f654c7ca2ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:55.277287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5be6f4-ef40-49e2-b6dd-c10cf2f770bb admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:55.283605 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf165f9-6025-400c-95ac-9409724e8a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:55.284224 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf165f9-6025-400c-95ac-9409724e8a8f admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.284555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf165f9-6025-400c-95ac-9409724e8a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.284833 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf165f9-6025-400c-95ac-9409724e8a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.285452 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf165f9-6025-400c-95ac-9409724e8a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:55.287914 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf165f9-6025-400c-95ac-9409724e8a8f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:55.288660 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:55.325103 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1218/2436] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:48:55 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:55.330031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf165f9-6025-400c-95ac-9409724e8a8f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:55.400559 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1219/2437] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:48:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:48:55.418846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf165f9-6025-400c-95ac-9409724e8a8f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:55.431602 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac5133-6fbe-48ee-9fa2-f3ada41bb010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:55.433342 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac5133-6fbe-48ee-9fa2-f3ada41bb010 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.433730 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac5133-6fbe-48ee-9fa2-f3ada41bb010 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.434106 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac5133-6fbe-48ee-9fa2-f3ada41bb010 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.434862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac5133-6fbe-48ee-9fa2-f3ada41bb010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:55.436949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac5133-6fbe-48ee-9fa2-f3ada41bb010 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:55.456590 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442fbc3-6d18-484d-975c-f654c7ca2ac5 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:55.459043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5442fbc3-6d18-484d-975c-f654c7ca2ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:55.464379 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1219/2438] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:48:55 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:48:55.467367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac5133-6fbe-48ee-9fa2-f3ada41bb010 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:55.478038 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac5133-6fbe-48ee-9fa2-f3ada41bb010 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:55.486878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b41ead-d332-4c09-a9f0-d373fd728595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:55.488333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b41ead-d332-4c09-a9f0-d373fd728595 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.488696 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b41ead-d332-4c09-a9f0-d373fd728595 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.488995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b41ead-d332-4c09-a9f0-d373fd728595 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.489654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b41ead-d332-4c09-a9f0-d373fd728595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:55.491297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b41ead-d332-4c09-a9f0-d373fd728595 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:55.510418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1220/2439] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:48:55.511167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b41ead-d332-4c09-a9f0-d373fd728595 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:55.523113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b41ead-d332-4c09-a9f0-d373fd728595 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:55.530353 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d59fa8-0ede-4518-9f53-1a13ad539596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:55.531544 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d59fa8-0ede-4518-9f53-1a13ad539596 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.532394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d59fa8-0ede-4518-9f53-1a13ad539596 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.532798 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d59fa8-0ede-4518-9f53-1a13ad539596 admin admin] PATH_INFO: `/v3/projects/c89e78079ce84be28aa59662ce0f198f/users/89fe55a50e8a487fb1013ee73f61ff84/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.550336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442fbc3-6d18-484d-975c-f654c7ca2ac5 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:55.552326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d59fa8-0ede-4518-9f53-1a13ad539596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89e78079ce84be28aa59662ce0f198f, user_id=89fe55a50e8a487fb1013ee73f61ff84, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:55.556320 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d59fa8-0ede-4518-9f53-1a13ad539596 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:55.573035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1221/2440] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:55 2022] PUT /identity/v3/projects/c89e78079ce84be28aa59662ce0f198f/users/89fe55a50e8a487fb1013ee73f61ff84/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:55.577039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d59fa8-0ede-4518-9f53-1a13ad539596 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:55.627088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086f697-813c-4020-933a-7c20c45847fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:55.627804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086f697-813c-4020-933a-7c20c45847fa service swift] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.628159 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086f697-813c-4020-933a-7c20c45847fa service swift] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.628479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086f697-813c-4020-933a-7c20c45847fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.658720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d59fa8-0ede-4518-9f53-1a13ad539596 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:55.664213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4e3d-4cd8-4cd8-ad3e-ed1f5970b918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:55.664857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4e3d-4cd8-4cd8-ad3e-ed1f5970b918 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.665140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4e3d-4cd8-4cd8-ad3e-ed1f5970b918 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.665434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4e3d-4cd8-4cd8-ad3e-ed1f5970b918 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.666082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a4e3d-4cd8-4cd8-ad3e-ed1f5970b918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:55.667682 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a4e3d-4cd8-4cd8-ad3e-ed1f5970b918 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:55.680522 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1222/2441] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:48:55.682409 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:55.683371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4e3d-4cd8-4cd8-ad3e-ed1f5970b918 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:55.696144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086f697-813c-4020-933a-7c20c45847fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:55.697798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086f697-813c-4020-933a-7c20c45847fa service swift] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:55.702585 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4e3d-4cd8-4cd8-ad3e-ed1f5970b918 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:55.708526 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0e445-3e27-4d9a-8bdd-a81505175005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:55.708886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1220/2442] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:48:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 14:48:55.709785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0e445-3e27-4d9a-8bdd-a81505175005 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.710555 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0e445-3e27-4d9a-8bdd-a81505175005 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.710747 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0e445-3e27-4d9a-8bdd-a81505175005 admin admin] PATH_INFO: `/v3/projects/c89e78079ce84be28aa59662ce0f198f/users/89fe55a50e8a487fb1013ee73f61ff84/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.712755 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0e445-3e27-4d9a-8bdd-a81505175005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89e78079ce84be28aa59662ce0f198f, user_id=89fe55a50e8a487fb1013ee73f61ff84, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:55.715363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0e445-3e27-4d9a-8bdd-a81505175005 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:55.736954 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1223/2443] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:48:55 2022] PUT /identity/v3/projects/c89e78079ce84be28aa59662ce0f198f/users/89fe55a50e8a487fb1013ee73f61ff84/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:48:55.747275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1daf1-4989-4933-b3b4-e664a664fac4 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:55.747562 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1daf1-4989-4933-b3b4-e664a664fac4 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:55.747843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1daf1-4989-4933-b3b4-e664a664fac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:55.780191 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-dba1daf1-4989-4933-b3b4-e664a664fac4 None None] MFA Rules not processed for user `89fe55a50e8a487fb1013ee73f61ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:55.782393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-dba1daf1-4989-4933-b3b4-e664a664fac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:55.887969 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1221/2444] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:55.904990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0e445-3e27-4d9a-8bdd-a81505175005 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:55.908161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9ff0e445-3e27-4d9a-8bdd-a81505175005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:55.935277 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:56.007734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0e445-3e27-4d9a-8bdd-a81505175005 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:56.018432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724a201-d70d-4eb5-b07f-38f89f36bb7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:56.019302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724a201-d70d-4eb5-b07f-38f89f36bb7c service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:56.019765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724a201-d70d-4eb5-b07f-38f89f36bb7c service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:56.020220 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724a201-d70d-4eb5-b07f-38f89f36bb7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:56.101630 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724a201-d70d-4eb5-b07f-38f89f36bb7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:56.103341 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724a201-d70d-4eb5-b07f-38f89f36bb7c service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:56.114398 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1224/2445] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:56.405955 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dc569-c019-48a0-a13f-86988d611a4a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:56.406852 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dc569-c019-48a0-a13f-86988d611a4a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:56.407117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dc569-c019-48a0-a13f-86988d611a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:56.445349 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-d68dc569-c019-48a0-a13f-86988d611a4a None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:56.446639 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d68dc569-c019-48a0-a13f-86988d611a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:56.523568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1222/2446] 10.222.0.13 () {58 vars in 967 bytes} [Thu Oct 20 14:48:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:56.537874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724a201-d70d-4eb5-b07f-38f89f36bb7c service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:56.605996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724a201-d70d-4eb5-b07f-38f89f36bb7c service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:56.611266 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59898f-dcd8-49fb-9da9-ac1c1d7b7260 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:56.611869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59898f-dcd8-49fb-9da9-ac1c1d7b7260 service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:56.612148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59898f-dcd8-49fb-9da9-ac1c1d7b7260 service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:56.612459 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59898f-dcd8-49fb-9da9-ac1c1d7b7260 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:56.620477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59898f-dcd8-49fb-9da9-ac1c1d7b7260 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:56.622059 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59898f-dcd8-49fb-9da9-ac1c1d7b7260 service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:56.630771 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1225/2447] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:48:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:48:56.852705 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7faaf-950e-4a98-99fc-07ea21a03c95 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:56.852993 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7faaf-950e-4a98-99fc-07ea21a03c95 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:56.853264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7faaf-950e-4a98-99fc-07ea21a03c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:56.871957 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-13d7faaf-950e-4a98-99fc-07ea21a03c95 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:56.873163 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-13d7faaf-950e-4a98-99fc-07ea21a03c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:56.880851 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1223/2448] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:48:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:56.892739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59898f-dcd8-49fb-9da9-ac1c1d7b7260 service swift] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:56.903005 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59898f-dcd8-49fb-9da9-ac1c1d7b7260 service swift] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:56.908521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000be469-98e8-4cb2-b063-747292a4f352 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:56.909131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000be469-98e8-4cb2-b063-747292a4f352 service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:56.909409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000be469-98e8-4cb2-b063-747292a4f352 service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:56.909680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000be469-98e8-4cb2-b063-747292a4f352 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:56.917431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000be469-98e8-4cb2-b063-747292a4f352 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:56.919027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000be469-98e8-4cb2-b063-747292a4f352 service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:56.927811 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1226/2449] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:48:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 14:48:57.267998 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e294-7423-490c-a8e7-c6c4b18ba2ad None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:57.268307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e294-7423-490c-a8e7-c6c4b18ba2ad None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:57.268609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e294-7423-490c-a8e7-c6c4b18ba2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:57.288500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-1c50e294-7423-490c-a8e7-c6c4b18ba2ad None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:48:57.289721 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-1c50e294-7423-490c-a8e7-c6c4b18ba2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:57.297188 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1224/2450] 10.222.0.13 () {58 vars in 967 bytes} [Thu Oct 20 14:48:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:48:57.309850 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000be469-98e8-4cb2-b063-747292a4f352 service swift] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:57.322360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000be469-98e8-4cb2-b063-747292a4f352 service swift] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:57.327369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16b5b0-c62c-41b0-bab5-45cd3ed6b7a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:57.328143 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16b5b0-c62c-41b0-bab5-45cd3ed6b7a3 service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:57.328703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16b5b0-c62c-41b0-bab5-45cd3ed6b7a3 service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:57.328805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16b5b0-c62c-41b0-bab5-45cd3ed6b7a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:57.337327 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16b5b0-c62c-41b0-bab5-45cd3ed6b7a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:57.339052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16b5b0-c62c-41b0-bab5-45cd3ed6b7a3 service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:57.349323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1227/2451] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:48:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:48:57.777264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50e294-7423-490c-a8e7-c6c4b18ba2ad None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:57.787503 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50e294-7423-490c-a8e7-c6c4b18ba2ad None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:57.792013 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d9362-0e3c-437b-b399-402c79b4f02c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:57.792664 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d9362-0e3c-437b-b399-402c79b4f02c service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:57.792948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d9362-0e3c-437b-b399-402c79b4f02c service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:57.793225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d9362-0e3c-437b-b399-402c79b4f02c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:57.862635 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:48:57.869643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d9362-0e3c-437b-b399-402c79b4f02c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:57.871834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d9362-0e3c-437b-b399-402c79b4f02c service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:57.896794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1225/2452] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:58.494688 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16b5b0-c62c-41b0-bab5-45cd3ed6b7a3 service swift] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:58.496183 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-ad16b5b0-c62c-41b0-bab5-45cd3ed6b7a3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:58.509292 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16b5b0-c62c-41b0-bab5-45cd3ed6b7a3 service swift] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:58.513288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2239c-199d-43cf-9550-0b0b422bb74d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:58.513888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2239c-199d-43cf-9550-0b0b422bb74d service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:58.514161 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2239c-199d-43cf-9550-0b0b422bb74d service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:58.514429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2239c-199d-43cf-9550-0b0b422bb74d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:58.525436 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2239c-199d-43cf-9550-0b0b422bb74d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:58.527687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2239c-199d-43cf-9550-0b0b422bb74d service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:58.538686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1228/2453] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:48:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:48:59.794087 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d9362-0e3c-437b-b399-402c79b4f02c service cinder] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:48:59.795524 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c33d9362-0e3c-437b-b399-402c79b4f02c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:48:59.865084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d9362-0e3c-437b-b399-402c79b4f02c service cinder] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:48:59.869823 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:48:59.870390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:48:59.870672 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:48:59.870949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] PATH_INFO: `/v3/users/182a86c0f20a4bd3bc7523cae7e829b3` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:48:59.872096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=182a86c0f20a4bd3bc7523cae7e829b3)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:48:59.874339 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:48:59.934553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182a86c0f20a4bd3bc7523cae7e829b3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:59.942287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182a86c0f20a4bd3bc7523cae7e829b3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:59.942982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182a86c0f20a4bd3bc7523cae7e829b3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:59.943595 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182a86c0f20a4bd3bc7523cae7e829b3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:59.954206 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182a86c0f20a4bd3bc7523cae7e829b3'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:48:59.993740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1226/2454] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:48:59 2022] DELETE /identity/v3/users/182a86c0f20a4bd3bc7523cae7e829b3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:48:59.999996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f0e7a-c19e-4a3b-b300-98bc35651642 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:00.000557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f0e7a-c19e-4a3b-b300-98bc35651642 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:00.000557 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f0e7a-c19e-4a3b-b300-98bc35651642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:00.028471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-438f0e7a-c19e-4a3b-b300-98bc35651642 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:00.030139 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-438f0e7a-c19e-4a3b-b300-98bc35651642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:00.052830 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:00.103796 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1229/2455] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:00.114306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:00.115728 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:00.196629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189fe71-79cf-4a3e-8f26-ed52688a9d57 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:00.201318 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c40c70-4610-4eae-975e-98579679fea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:00.202017 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c40c70-4610-4eae-975e-98579679fea6 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:00.202297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c40c70-4610-4eae-975e-98579679fea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:00.203052 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c40c70-4610-4eae-975e-98579679fea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:00.214051 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c40c70-4610-4eae-975e-98579679fea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:00.215982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c40c70-4610-4eae-975e-98579679fea6 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:00.227475 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1227/2456] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:00.439532 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f0e7a-c19e-4a3b-b300-98bc35651642 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:00.441151 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-438f0e7a-c19e-4a3b-b300-98bc35651642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:00.463354 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f0e7a-c19e-4a3b-b300-98bc35651642 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:00.482920 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:00.483769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:00.484169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:00.484487 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] PATH_INFO: `/v3/projects/76c13daa80f7403298c110265b3539a3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:00.485829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c13daa80f7403298c110265b3539a3)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:00.487750 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:00.556551 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c13daa80f7403298c110265b3539a3', 'request_id': 'req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:00.563343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c13daa80f7403298c110265b3539a3', 'request_id': 'req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:00.571596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c13daa80f7403298c110265b3539a3', 'request_id': 'req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:00.584713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] The token cache is being invalidate because project 76c13daa80f7403298c110265b3539a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:00.585067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:00.585815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-8884de6e-a45f-4e3c-a43a-2ed7c8136f7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:00.587434 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1230/2457] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:00 2022] DELETE /identity/v3/projects/76c13daa80f7403298c110265b3539a3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:02.485085 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c40c70-4610-4eae-975e-98579679fea6 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:02.486517 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c4c40c70-4610-4eae-975e-98579679fea6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:02.500920 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c40c70-4610-4eae-975e-98579679fea6 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:02.513050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:02.560621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:02.561256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:02.561530 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:02.561901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] PATH_INFO: `/v3/users/f240bb91a0f84ba6a742788f77f79082` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:02.563726 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f240bb91a0f84ba6a742788f77f79082)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:02.565734 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:02.612671 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f240bb91a0f84ba6a742788f77f79082'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:02.619413 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f240bb91a0f84ba6a742788f77f79082'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:02.620068 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f240bb91a0f84ba6a742788f77f79082'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:02.620677 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f240bb91a0f84ba6a742788f77f79082'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:02.629006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f240bb91a0f84ba6a742788f77f79082'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:02.646057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1228/2458] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:02 2022] DELETE /identity/v3/users/f240bb91a0f84ba6a742788f77f79082 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:02.651014 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98fb4d-54aa-462f-9d06-03f0f9364d02 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:02.651520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98fb4d-54aa-462f-9d06-03f0f9364d02 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:02.651629 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98fb4d-54aa-462f-9d06-03f0f9364d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:02.673125 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-be98fb4d-54aa-462f-9d06-03f0f9364d02 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:02.674332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-be98fb4d-54aa-462f-9d06-03f0f9364d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:02.744662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1231/2459] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:02.753106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:02.754977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:02.825495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9300140-b9bd-427f-9b95-33adb2a85855 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:02.829324 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dea2f3-77dc-4e6a-a98e-02499d24f05c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:02.830056 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea2f3-77dc-4e6a-a98e-02499d24f05c service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:02.830358 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea2f3-77dc-4e6a-a98e-02499d24f05c service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:02.830643 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea2f3-77dc-4e6a-a98e-02499d24f05c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:02.841256 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dea2f3-77dc-4e6a-a98e-02499d24f05c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:02.842708 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dea2f3-77dc-4e6a-a98e-02499d24f05c service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:02.853285 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1229/2460] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:03.129620 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98fb4d-54aa-462f-9d06-03f0f9364d02 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:03.131144 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-be98fb4d-54aa-462f-9d06-03f0f9364d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:03.143188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98fb4d-54aa-462f-9d06-03f0f9364d02 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:03.148673 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:03.149332 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:03.152308 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:03.152680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] PATH_INFO: `/v3/projects/4099c8c7a4e44711832de4e19e2f8b81` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:03.153976 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4099c8c7a4e44711832de4e19e2f8b81)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:03.155986 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:03.196361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:03.206992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4099c8c7a4e44711832de4e19e2f8b81', 'request_id': 'req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.213006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4099c8c7a4e44711832de4e19e2f8b81', 'request_id': 'req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.221131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4099c8c7a4e44711832de4e19e2f8b81', 'request_id': 'req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.233363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] The token cache is being invalidate because project 4099c8c7a4e44711832de4e19e2f8b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:03.234194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.235050 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.236952 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1232/2461] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:03 2022] DELETE /identity/v3/projects/4099c8c7a4e44711832de4e19e2f8b81 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:03.241077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dea2f3-77dc-4e6a-a98e-02499d24f05c service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:03.251506 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dea2f3-77dc-4e6a-a98e-02499d24f05c service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:03.291380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:03.314469 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:03.315118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:03.315401 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:03.315678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] PATH_INFO: `/v3/users/312262ba49ec4679b2868a05cdb517ed` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:03.316886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=312262ba49ec4679b2868a05cdb517ed)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:03.318946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:03.372495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312262ba49ec4679b2868a05cdb517ed'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.380162 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312262ba49ec4679b2868a05cdb517ed'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.380850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312262ba49ec4679b2868a05cdb517ed'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.381447 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312262ba49ec4679b2868a05cdb517ed'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.389760 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312262ba49ec4679b2868a05cdb517ed'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.408241 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1230/2462] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:03 2022] DELETE /identity/v3/users/312262ba49ec4679b2868a05cdb517ed => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:03.703609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:03.705148 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:03.719366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33b5dd-fb89-4e71-89ca-d43c2dc4a56b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:03.784666 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:03.785313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:03.785596 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:03.785872 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] PATH_INFO: `/v3/projects/bcfa46ad41fc4ce1a6d1da8b1352499b` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:03.786990 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcfa46ad41fc4ce1a6d1da8b1352499b)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:03.788932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:03.836820 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcfa46ad41fc4ce1a6d1da8b1352499b', 'request_id': 'req-090dcb13-c172-456c-b8ac-58a509f6a625'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.842675 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcfa46ad41fc4ce1a6d1da8b1352499b', 'request_id': 'req-090dcb13-c172-456c-b8ac-58a509f6a625'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.852066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcfa46ad41fc4ce1a6d1da8b1352499b', 'request_id': 'req-090dcb13-c172-456c-b8ac-58a509f6a625'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.866323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] The token cache is being invalidate because project bcfa46ad41fc4ce1a6d1da8b1352499b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:03.866613 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.867263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:03.869205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1233/2463] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:03 2022] DELETE /identity/v3/projects/bcfa46ad41fc4ce1a6d1da8b1352499b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:49:03.873093 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:03.884092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:03.898509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96430dea-12cc-4e4e-97ac-4f4c70faa3c1 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:03.929590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:03.931217 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:03.963693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:03.964509 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:03.965243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:03.965560 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] PATH_INFO: `/v3/users/89fe55a50e8a487fb1013ee73f61ff84` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:03.966754 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89fe55a50e8a487fb1013ee73f61ff84)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:03.969881 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:03.996492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:04.032418 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89fe55a50e8a487fb1013ee73f61ff84'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.040724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89fe55a50e8a487fb1013ee73f61ff84'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.041434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89fe55a50e8a487fb1013ee73f61ff84'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.042137 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89fe55a50e8a487fb1013ee73f61ff84'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.043126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090dcb13-c172-456c-b8ac-58a509f6a625 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:04.046573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:04.050150 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c8574-8e18-485a-99a0-23e93cd6789b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc18a78229064e8d8ea4321c3c23c4ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:04.050923 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c8574-8e18-485a-99a0-23e93cd6789b service placement] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:04.051246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c8574-8e18-485a-99a0-23e93cd6789b service placement] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:04.051572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c8574-8e18-485a-99a0-23e93cd6789b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:04.053301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89fe55a50e8a487fb1013ee73f61ff84'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.053546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a56c8574-8e18-485a-99a0-23e93cd6789b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:04.067528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c8574-8e18-485a-99a0-23e93cd6789b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:04.069202 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c8574-8e18-485a-99a0-23e93cd6789b service placement] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:04.081824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1231/2464] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:03 2022] DELETE /identity/v3/users/89fe55a50e8a487fb1013ee73f61ff84 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:04.085415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1234/2465] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:49:04.401265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:04.402821 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:04.505850 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cbeff-1784-487a-848a-d82a1fa7dc40 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:04.511289 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:04.512033 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:04.512322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:04.512631 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] PATH_INFO: `/v3/projects/c89e78079ce84be28aa59662ce0f198f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:04.513764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89e78079ce84be28aa59662ce0f198f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:04.516556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:04.570088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89e78079ce84be28aa59662ce0f198f', 'request_id': 'req-50779b64-cc67-4951-a2f9-d8de41bb0f75'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.575494 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89e78079ce84be28aa59662ce0f198f', 'request_id': 'req-50779b64-cc67-4951-a2f9-d8de41bb0f75'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.584306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89e78079ce84be28aa59662ce0f198f', 'request_id': 'req-50779b64-cc67-4951-a2f9-d8de41bb0f75'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.596338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] The token cache is being invalidate because project c89e78079ce84be28aa59662ce0f198f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:04.596650 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.597283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-50779b64-cc67-4951-a2f9-d8de41bb0f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:04.598775 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1232/2466] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:04 2022] DELETE /identity/v3/projects/c89e78079ce84be28aa59662ce0f198f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:04.604196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74b18f-88a5-44a7-9bda-23652d323a52 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:04.604513 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74b18f-88a5-44a7-9bda-23652d323a52 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:04.604789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74b18f-88a5-44a7-9bda-23652d323a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:04.625556 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-0d74b18f-88a5-44a7-9bda-23652d323a52 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:04.626749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-0d74b18f-88a5-44a7-9bda-23652d323a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:04.685637 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1235/2467] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:49:04.693686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857e45-1e4f-4ccd-9df0-fae75a210363 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:04.693977 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857e45-1e4f-4ccd-9df0-fae75a210363 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:04.694251 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857e45-1e4f-4ccd-9df0-fae75a210363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:04.714059 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-af857e45-1e4f-4ccd-9df0-fae75a210363 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:04.715281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-af857e45-1e4f-4ccd-9df0-fae75a210363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:04.724566 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1233/2468] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:04.728924 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74b18f-88a5-44a7-9bda-23652d323a52 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:04.739080 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74b18f-88a5-44a7-9bda-23652d323a52 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:04.744983 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c8e67-ab50-4b76-9399-53a639c17ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:04.745580 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c8e67-ab50-4b76-9399-53a639c17ddd admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:04.745857 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c8e67-ab50-4b76-9399-53a639c17ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:04.746126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c8e67-ab50-4b76-9399-53a639c17ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:04.746766 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c8e67-ab50-4b76-9399-53a639c17ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:49:04.747066 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c8e67-ab50-4b76-9399-53a639c17ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:04.748702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c8e67-ab50-4b76-9399-53a639c17ddd admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:04.765900 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1236/2469] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:49:04.767168 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af857e45-1e4f-4ccd-9df0-fae75a210363 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:04.776263 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af857e45-1e4f-4ccd-9df0-fae75a210363 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:04.781568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2db183-8b82-4b30-9f39-2a7d5503885a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:04.782161 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2db183-8b82-4b30-9f39-2a7d5503885a admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:04.782438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2db183-8b82-4b30-9f39-2a7d5503885a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:04.782709 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2db183-8b82-4b30-9f39-2a7d5503885a admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:04.783365 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2db183-8b82-4b30-9f39-2a7d5503885a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:04.785306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2db183-8b82-4b30-9f39-2a7d5503885a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:04.813088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1234/2470] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:49:04 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:04.817528 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c8e67-ab50-4b76-9399-53a639c17ddd admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:04.829674 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c8e67-ab50-4b76-9399-53a639c17ddd admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:04.889933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea9f1-c88c-457d-bca1-118b43659d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:04.890684 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea9f1-c88c-457d-bca1-118b43659d3f admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:04.891121 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea9f1-c88c-457d-bca1-118b43659d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:04.891399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea9f1-c88c-457d-bca1-118b43659d3f admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:04.892067 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea9f1-c88c-457d-bca1-118b43659d3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:04.894010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea9f1-c88c-457d-bca1-118b43659d3f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:04.917259 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1237/2471] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:49:04 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:49:04.922252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2db183-8b82-4b30-9f39-2a7d5503885a admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:04.945610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2db183-8b82-4b30-9f39-2a7d5503885a admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:04.945610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaccdf-01f4-4359-9570-91f9854303f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:04.945610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaccdf-01f4-4359-9570-91f9854303f2 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:04.945610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaccdf-01f4-4359-9570-91f9854303f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:04.946001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaccdf-01f4-4359-9570-91f9854303f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:04.946001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaccdf-01f4-4359-9570-91f9854303f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:04.946001 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaccdf-01f4-4359-9570-91f9854303f2 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:04.955807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1235/2472] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:04.959842 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea9f1-c88c-457d-bca1-118b43659d3f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:04.970432 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea9f1-c88c-457d-bca1-118b43659d3f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:04.975571 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217745c-4269-4f85-afb8-0a32fe56b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:04.976194 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217745c-4269-4f85-afb8-0a32fe56b280 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:04.976533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217745c-4269-4f85-afb8-0a32fe56b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:04.976834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217745c-4269-4f85-afb8-0a32fe56b280 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:04.977713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217745c-4269-4f85-afb8-0a32fe56b280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:04.979323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217745c-4269-4f85-afb8-0a32fe56b280 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:04.990786 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1238/2473] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:04.995167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaccdf-01f4-4359-9570-91f9854303f2 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:05.006646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaccdf-01f4-4359-9570-91f9854303f2 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:05.011875 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962f5e1-bb9e-495d-91cb-7ced66beb7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:05.012565 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962f5e1-bb9e-495d-91cb-7ced66beb7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:05.012849 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962f5e1-bb9e-495d-91cb-7ced66beb7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:05.013167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962f5e1-bb9e-495d-91cb-7ced66beb7f8 admin admin] PATH_INFO: `/v3/projects/7bafa4c5aa6943f9a860f577c957f33d/users/57c1b64154514e43aec44f21af65cff7/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:05.028260 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962f5e1-bb9e-495d-91cb-7ced66beb7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bafa4c5aa6943f9a860f577c957f33d, user_id=57c1b64154514e43aec44f21af65cff7, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:05.031498 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962f5e1-bb9e-495d-91cb-7ced66beb7f8 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:05.044218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1236/2474] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:49:04 2022] PUT /identity/v3/projects/7bafa4c5aa6943f9a860f577c957f33d/users/57c1b64154514e43aec44f21af65cff7/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:49:05.050378 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a915a71-cb16-4320-a8ee-08c148b7dc86 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:05.050664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a915a71-cb16-4320-a8ee-08c148b7dc86 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:05.050938 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a915a71-cb16-4320-a8ee-08c148b7dc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:05.069299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-2a915a71-cb16-4320-a8ee-08c148b7dc86 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:05.070561 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-2a915a71-cb16-4320-a8ee-08c148b7dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:05.097619 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:05.132510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1239/2475] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:05.143121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962f5e1-bb9e-495d-91cb-7ced66beb7f8 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:05.144892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e962f5e1-bb9e-495d-91cb-7ced66beb7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:05.151539 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:05.205801 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962f5e1-bb9e-495d-91cb-7ced66beb7f8 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:05.209434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad479c-b887-4d7d-a9a3-955c6c7eae54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:05.210043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad479c-b887-4d7d-a9a3-955c6c7eae54 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:05.210325 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad479c-b887-4d7d-a9a3-955c6c7eae54 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:05.210602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad479c-b887-4d7d-a9a3-955c6c7eae54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:05.220412 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad479c-b887-4d7d-a9a3-955c6c7eae54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:05.221862 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad479c-b887-4d7d-a9a3-955c6c7eae54 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:05.232184 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1237/2476] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:05.879000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164449ab-e591-47a1-abcd-c1ea17d4864e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:05.879507 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164449ab-e591-47a1-abcd-c1ea17d4864e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:05.879615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164449ab-e591-47a1-abcd-c1ea17d4864e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:05.886716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad479c-b887-4d7d-a9a3-955c6c7eae54 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:05.888467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-a6ad479c-b887-4d7d-a9a3-955c6c7eae54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:05.905942 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad479c-b887-4d7d-a9a3-955c6c7eae54 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:05.911407 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:05.911597 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-164449ab-e591-47a1-abcd-c1ea17d4864e None None] MFA Rules not processed for user `57c1b64154514e43aec44f21af65cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:05.912119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:05.912585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:05.912934 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] PATH_INFO: `/v3/users/c08c28e36d5a4f05847e2940efdf6b7c` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:05.913076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-164449ab-e591-47a1-abcd-c1ea17d4864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:05.914166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08c28e36d5a4f05847e2940efdf6b7c)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:05.916380 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:05.978520 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08c28e36d5a4f05847e2940efdf6b7c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:05.988904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08c28e36d5a4f05847e2940efdf6b7c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:05.989923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08c28e36d5a4f05847e2940efdf6b7c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:05.991765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08c28e36d5a4f05847e2940efdf6b7c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.020196 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e3d8fb82-fa06-4079-9706-68796cfb417d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08c28e36d5a4f05847e2940efdf6b7c'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.027499 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1240/2477] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:06.032566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164449ab-e591-47a1-abcd-c1ea17d4864e None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:06.037163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-164449ab-e591-47a1-abcd-c1ea17d4864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:06.046972 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1238/2478] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:05 2022] DELETE /identity/v3/users/c08c28e36d5a4f05847e2940efdf6b7c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:49:06.052763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fa855-4990-415e-9dde-c3f5b87d08a7 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:06.053275 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fa855-4990-415e-9dde-c3f5b87d08a7 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:06.053809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fa855-4990-415e-9dde-c3f5b87d08a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:06.081084 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-272fa855-4990-415e-9dde-c3f5b87d08a7 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:06.083209 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-272fa855-4990-415e-9dde-c3f5b87d08a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:06.108053 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164449ab-e591-47a1-abcd-c1ea17d4864e None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:06.151097 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1239/2479] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:06.159265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce66f4-fd3a-4efe-9b81-269b40de6497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:06.160004 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce66f4-fd3a-4efe-9b81-269b40de6497 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:06.160288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce66f4-fd3a-4efe-9b81-269b40de6497 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:06.160668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce66f4-fd3a-4efe-9b81-269b40de6497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:06.162178 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fa855-4990-415e-9dde-c3f5b87d08a7 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:06.163594 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-272fa855-4990-415e-9dde-c3f5b87d08a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:06.188131 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:06.195222 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=100 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:06.230113 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fa855-4990-415e-9dde-c3f5b87d08a7 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:06.230347 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce66f4-fd3a-4efe-9b81-269b40de6497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:06.231678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce66f4-fd3a-4efe-9b81-269b40de6497 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:06.234297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285e89-59b9-4d4d-a3fc-b26c799e827c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:06.234909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285e89-59b9-4d4d-a3fc-b26c799e827c service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:06.235186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285e89-59b9-4d4d-a3fc-b26c799e827c service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:06.235456 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285e89-59b9-4d4d-a3fc-b26c799e827c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:06.242367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1241/2480] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:06.245031 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14285e89-59b9-4d4d-a3fc-b26c799e827c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:06.246485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14285e89-59b9-4d4d-a3fc-b26c799e827c service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:06.259796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1240/2481] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:49:06.558972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce66f4-fd3a-4efe-9b81-269b40de6497 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:06.560324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-84ce66f4-fd3a-4efe-9b81-269b40de6497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:06.580717 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce66f4-fd3a-4efe-9b81-269b40de6497 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:06.589288 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:06.590021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:06.590377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:06.590719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] PATH_INFO: `/v3/projects/2bb4f76c9f0247eeac4cc180dd5a11c0` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:06.592494 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb4f76c9f0247eeac4cc180dd5a11c0)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:06.595325 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:06.658679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb4f76c9f0247eeac4cc180dd5a11c0', 'request_id': 'req-25f8f3cb-8fe8-4356-863c-416ab303ed98'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.666627 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb4f76c9f0247eeac4cc180dd5a11c0', 'request_id': 'req-25f8f3cb-8fe8-4356-863c-416ab303ed98'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.677687 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb4f76c9f0247eeac4cc180dd5a11c0', 'request_id': 'req-25f8f3cb-8fe8-4356-863c-416ab303ed98'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.691126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] The token cache is being invalidate because project 2bb4f76c9f0247eeac4cc180dd5a11c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:06.691710 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.692427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.694037 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1242/2482] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:06 2022] DELETE /identity/v3/projects/2bb4f76c9f0247eeac4cc180dd5a11c0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:06.698473 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285e89-59b9-4d4d-a3fc-b26c799e827c service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:06.710299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285e89-59b9-4d4d-a3fc-b26c799e827c service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:06.770796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:06.771527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:06.771811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:06.772096 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] PATH_INFO: `/v3/users/45839c45fa0f4aa0874f044af1abaab2` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:06.773336 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45839c45fa0f4aa0874f044af1abaab2)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:06.775328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:06.822989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45839c45fa0f4aa0874f044af1abaab2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.830456 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45839c45fa0f4aa0874f044af1abaab2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.831140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45839c45fa0f4aa0874f044af1abaab2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.831841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45839c45fa0f4aa0874f044af1abaab2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.839792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-46d5809d-6579-424b-bf35-79f146bec694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45839c45fa0f4aa0874f044af1abaab2'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:06.860712 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1241/2483] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:06 2022] DELETE /identity/v3/users/45839c45fa0f4aa0874f044af1abaab2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:07.158300 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:07.159880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:07.174567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8f3cb-8fe8-4356-863c-416ab303ed98 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:07.238200 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:07.238834 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.239114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.239516 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] PATH_INFO: `/v3/projects/b7d216ac0c2247d39fd4970dde945fe5` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.240702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d216ac0c2247d39fd4970dde945fe5)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:07.245716 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:07.294520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d216ac0c2247d39fd4970dde945fe5', 'request_id': 'req-108294be-f4ff-4382-a7f8-d8a120508fdd'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:07.300118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:07.301662 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d216ac0c2247d39fd4970dde945fe5', 'request_id': 'req-108294be-f4ff-4382-a7f8-d8a120508fdd'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:07.311239 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d216ac0c2247d39fd4970dde945fe5', 'request_id': 'req-108294be-f4ff-4382-a7f8-d8a120508fdd'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:07.323358 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] The token cache is being invalidate because project b7d216ac0c2247d39fd4970dde945fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:07.323647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:07.324311 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-108294be-f4ff-4382-a7f8-d8a120508fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:07.325917 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1243/2484] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:07 2022] DELETE /identity/v3/projects/b7d216ac0c2247d39fd4970dde945fe5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:07.331716 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989f3b2-95c1-466b-8103-5cca6d3ede8a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.332310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989f3b2-95c1-466b-8103-5cca6d3ede8a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.332310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989f3b2-95c1-466b-8103-5cca6d3ede8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.353416 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-7989f3b2-95c1-466b-8103-5cca6d3ede8a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:07.354580 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7989f3b2-95c1-466b-8103-5cca6d3ede8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:07.394385 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:07.416599 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1242/2485] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:07.424740 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c0af0-dbdf-4c54-93b3-0e27d94575de None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.425027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c0af0-dbdf-4c54-93b3-0e27d94575de None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.425328 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c0af0-dbdf-4c54-93b3-0e27d94575de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.446057 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-9a7c0af0-dbdf-4c54-93b3-0e27d94575de None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:07.447360 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9a7c0af0-dbdf-4c54-93b3-0e27d94575de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:07.456512 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1244/2486] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:07.460713 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989f3b2-95c1-466b-8103-5cca6d3ede8a None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:07.470548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989f3b2-95c1-466b-8103-5cca6d3ede8a None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:07.475067 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f55907-5238-4080-9b1c-d256b13ee181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:07.475658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f55907-5238-4080-9b1c-d256b13ee181 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.475937 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f55907-5238-4080-9b1c-d256b13ee181 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.476214 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f55907-5238-4080-9b1c-d256b13ee181 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.477006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f55907-5238-4080-9b1c-d256b13ee181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:49:07.477589 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f55907-5238-4080-9b1c-d256b13ee181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:07.479218 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f55907-5238-4080-9b1c-d256b13ee181 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:07.491797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1243/2487] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:49:07.496185 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c0af0-dbdf-4c54-93b3-0e27d94575de None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:07.506079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c0af0-dbdf-4c54-93b3-0e27d94575de None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:07.510840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92db6b-916c-464b-ad60-ed754ad6a5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:07.511435 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92db6b-916c-464b-ad60-ed754ad6a5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.511715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92db6b-916c-464b-ad60-ed754ad6a5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.511988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92db6b-916c-464b-ad60-ed754ad6a5c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.512653 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92db6b-916c-464b-ad60-ed754ad6a5c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:07.514773 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92db6b-916c-464b-ad60-ed754ad6a5c5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:07.541867 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1245/2488] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:49:07 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:07.547185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f55907-5238-4080-9b1c-d256b13ee181 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:07.557880 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f55907-5238-4080-9b1c-d256b13ee181 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:07.620053 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db27e8-1b36-47aa-a9bf-559980dda58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:07.620701 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db27e8-1b36-47aa-a9bf-559980dda58f admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.620980 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db27e8-1b36-47aa-a9bf-559980dda58f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.621252 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db27e8-1b36-47aa-a9bf-559980dda58f admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.621891 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db27e8-1b36-47aa-a9bf-559980dda58f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:07.624962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db27e8-1b36-47aa-a9bf-559980dda58f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:07.647207 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1244/2489] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:49:07 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:07.651172 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92db6b-916c-464b-ad60-ed754ad6a5c5 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:07.662767 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92db6b-916c-464b-ad60-ed754ad6a5c5 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:07.672694 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d627ae-c9de-461c-bf75-ac21d4edf6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:07.673383 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d627ae-c9de-461c-bf75-ac21d4edf6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.673732 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d627ae-c9de-461c-bf75-ac21d4edf6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.674076 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d627ae-c9de-461c-bf75-ac21d4edf6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.674788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d627ae-c9de-461c-bf75-ac21d4edf6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:07.676494 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d627ae-c9de-461c-bf75-ac21d4edf6d0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:07.688444 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebca671-d314-412c-bb01-a77e87c1d4af None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.688730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebca671-d314-412c-bb01-a77e87c1d4af None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.689009 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebca671-d314-412c-bb01-a77e87c1d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.691782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1246/2490] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:49:07.693013 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d627ae-c9de-461c-bf75-ac21d4edf6d0 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:07.703351 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d627ae-c9de-461c-bf75-ac21d4edf6d0 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:07.708710 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-cebca671-d314-412c-bb01-a77e87c1d4af None None] MFA Rules not processed for user `57c1b64154514e43aec44f21af65cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:07.709636 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a738b-49f9-4e96-9aeb-657d062eb19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:07.709954 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-cebca671-d314-412c-bb01-a77e87c1d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:07.710248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a738b-49f9-4e96-9aeb-657d062eb19f admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.710552 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a738b-49f9-4e96-9aeb-657d062eb19f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.710823 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a738b-49f9-4e96-9aeb-657d062eb19f admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.711468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a738b-49f9-4e96-9aeb-657d062eb19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:07.713104 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a738b-49f9-4e96-9aeb-657d062eb19f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:07.723603 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1247/2491] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:49:07.727868 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a738b-49f9-4e96-9aeb-657d062eb19f admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:07.741821 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a738b-49f9-4e96-9aeb-657d062eb19f admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:07.748653 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cb5d6-9274-40e2-86c7-bd366185aaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:07.750374 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb5d6-9274-40e2-86c7-bd366185aaa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.750840 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb5d6-9274-40e2-86c7-bd366185aaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.751416 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb5d6-9274-40e2-86c7-bd366185aaa2 admin admin] PATH_INFO: `/v3/projects/bd68f72a421e464f9a1a09e8d78f610d/users/3df5c8f6291c4830bdfc7ca20413e5ba/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.775695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cb5d6-9274-40e2-86c7-bd366185aaa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd68f72a421e464f9a1a09e8d78f610d, user_id=3df5c8f6291c4830bdfc7ca20413e5ba, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:07.780655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cb5d6-9274-40e2-86c7-bd366185aaa2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:07.782658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1245/2492] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:49:07.798133 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1248/2493] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:49:07 2022] PUT /identity/v3/projects/bd68f72a421e464f9a1a09e8d78f610d/users/3df5c8f6291c4830bdfc7ca20413e5ba/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:49:07.809453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cae3dd-52e0-40e2-97c3-ad24151692f3 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:07.809737 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cae3dd-52e0-40e2-97c3-ad24151692f3 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:07.810046 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cae3dd-52e0-40e2-97c3-ad24151692f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:07.841253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-b0cae3dd-52e0-40e2-97c3-ad24151692f3 None None] MFA Rules not processed for user `3df5c8f6291c4830bdfc7ca20413e5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:07.842448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b0cae3dd-52e0-40e2-97c3-ad24151692f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:07.906166 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:07.926220 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1246/2494] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:49:07.927405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cb5d6-9274-40e2-86c7-bd366185aaa2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:07.993795 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cb5d6-9274-40e2-86c7-bd366185aaa2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:07.998761 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3d978-3f29-4780-8161-367c8406d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:07.999429 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3d978-3f29-4780-8161-367c8406d52e admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:08.000044 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3d978-3f29-4780-8161-367c8406d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:08.000427 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3d978-3f29-4780-8161-367c8406d52e admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:08.001101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3d978-3f29-4780-8161-367c8406d52e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:08.003025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3d978-3f29-4780-8161-367c8406d52e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:08.035680 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1249/2495] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:49:07 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:08.037420 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cae3dd-52e0-40e2-97c3-ad24151692f3 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:08.049007 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cae3dd-52e0-40e2-97c3-ad24151692f3 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:08.116175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ab6e7-2aba-437b-9dd0-25f73408c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:08.116835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ab6e7-2aba-437b-9dd0-25f73408c4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:08.117120 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ab6e7-2aba-437b-9dd0-25f73408c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:08.117396 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ab6e7-2aba-437b-9dd0-25f73408c4e9 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:08.118140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ab6e7-2aba-437b-9dd0-25f73408c4e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:08.120223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ab6e7-2aba-437b-9dd0-25f73408c4e9 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:08.151766 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1247/2496] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:49:08 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:49:08.155369 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3d978-3f29-4780-8161-367c8406d52e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:08.166023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3d978-3f29-4780-8161-367c8406d52e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:08.171263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa55c4-7375-4ee8-851a-5747ccf92b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:08.171851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa55c4-7375-4ee8-851a-5747ccf92b9d admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:08.172167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa55c4-7375-4ee8-851a-5747ccf92b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:08.172691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa55c4-7375-4ee8-851a-5747ccf92b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:08.173110 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa55c4-7375-4ee8-851a-5747ccf92b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:08.174731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa55c4-7375-4ee8-851a-5747ccf92b9d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:08.189758 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1250/2497] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:08.191036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ab6e7-2aba-437b-9dd0-25f73408c4e9 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:08.201106 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ab6e7-2aba-437b-9dd0-25f73408c4e9 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:08.207351 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea652df-f49b-48ba-8e8a-077d776e44c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:08.207816 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea652df-f49b-48ba-8e8a-077d776e44c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:08.208167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea652df-f49b-48ba-8e8a-077d776e44c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:08.208670 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea652df-f49b-48ba-8e8a-077d776e44c3 admin admin] PATH_INFO: `/v3/projects/fce42580a65b44a09c1bf06ec0036317/users/0c9fdd80423d4348a0df1d6ddd119c2f/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:08.227557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea652df-f49b-48ba-8e8a-077d776e44c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce42580a65b44a09c1bf06ec0036317, user_id=0c9fdd80423d4348a0df1d6ddd119c2f, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:08.230523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea652df-f49b-48ba-8e8a-077d776e44c3 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:08.245553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1248/2498] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:49:08 2022] PUT /identity/v3/projects/fce42580a65b44a09c1bf06ec0036317/users/0c9fdd80423d4348a0df1d6ddd119c2f/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:08.249962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa55c4-7375-4ee8-851a-5747ccf92b9d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:08.261972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa55c4-7375-4ee8-851a-5747ccf92b9d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:08.306705 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:08.326659 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7f4c8-dd65-4688-b300-222b9bbfd108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:08.327502 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7f4c8-dd65-4688-b300-222b9bbfd108 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:08.327787 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7f4c8-dd65-4688-b300-222b9bbfd108 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:08.329315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7f4c8-dd65-4688-b300-222b9bbfd108 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:08.330584 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7f4c8-dd65-4688-b300-222b9bbfd108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:08.332649 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7f4c8-dd65-4688-b300-222b9bbfd108 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:08.344025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1251/2499] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:08.349417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea652df-f49b-48ba-8e8a-077d776e44c3 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:08.365167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea652df-f49b-48ba-8e8a-077d776e44c3 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:08.369458 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be83dd1-e332-4ab4-a4cb-312d306f0f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:08.370118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be83dd1-e332-4ab4-a4cb-312d306f0f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:08.370550 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be83dd1-e332-4ab4-a4cb-312d306f0f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:08.370834 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be83dd1-e332-4ab4-a4cb-312d306f0f89 admin admin] PATH_INFO: `/v3/projects/fce42580a65b44a09c1bf06ec0036317/users/0c9fdd80423d4348a0df1d6ddd119c2f/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:08.373186 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be83dd1-e332-4ab4-a4cb-312d306f0f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce42580a65b44a09c1bf06ec0036317, user_id=0c9fdd80423d4348a0df1d6ddd119c2f, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:08.379307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be83dd1-e332-4ab4-a4cb-312d306f0f89 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:08.392951 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1249/2500] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:49:08 2022] PUT /identity/v3/projects/fce42580a65b44a09c1bf06ec0036317/users/0c9fdd80423d4348a0df1d6ddd119c2f/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:08.403395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c69f29-6f1d-40c1-836c-247f14171ae8 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:08.403589 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c69f29-6f1d-40c1-836c-247f14171ae8 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:08.404211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c69f29-6f1d-40c1-836c-247f14171ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:08.436229 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-04c69f29-6f1d-40c1-836c-247f14171ae8 None None] MFA Rules not processed for user `0c9fdd80423d4348a0df1d6ddd119c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:08.437615 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-04c69f29-6f1d-40c1-836c-247f14171ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:08.516246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1252/2501] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:08.524308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be83dd1-e332-4ab4-a4cb-312d306f0f89 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:08.526229 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8be83dd1-e332-4ab4-a4cb-312d306f0f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:08.590638 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be83dd1-e332-4ab4-a4cb-312d306f0f89 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:08.595499 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5933fb-917d-4e63-85de-30d04ecee32b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:08.596136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5933fb-917d-4e63-85de-30d04ecee32b service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:08.596426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5933fb-917d-4e63-85de-30d04ecee32b service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:08.596736 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5933fb-917d-4e63-85de-30d04ecee32b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:08.617674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5933fb-917d-4e63-85de-30d04ecee32b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:08.619232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5933fb-917d-4e63-85de-30d04ecee32b service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:08.629557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1250/2502] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:09.031027 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c69f29-6f1d-40c1-836c-247f14171ae8 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:09.097663 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c69f29-6f1d-40c1-836c-247f14171ae8 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:09.102521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:09.103147 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:09.103425 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:09.103706 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] PATH_INFO: `/v3/users/3df5c8f6291c4830bdfc7ca20413e5ba` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:09.104869 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3df5c8f6291c4830bdfc7ca20413e5ba)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:09.106815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:09.159616 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3df5c8f6291c4830bdfc7ca20413e5ba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.167534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3df5c8f6291c4830bdfc7ca20413e5ba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.168400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3df5c8f6291c4830bdfc7ca20413e5ba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.169114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3df5c8f6291c4830bdfc7ca20413e5ba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.178315 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3df5c8f6291c4830bdfc7ca20413e5ba'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.183421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:09.202578 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1253/2503] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:09 2022] DELETE /identity/v3/users/3df5c8f6291c4830bdfc7ca20413e5ba => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:09.207362 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66960a5b-91ee-4b59-9f5b-d63f3ed920eb None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:09.207633 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66960a5b-91ee-4b59-9f5b-d63f3ed920eb None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:09.207911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66960a5b-91ee-4b59-9f5b-d63f3ed920eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:09.228197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-66960a5b-91ee-4b59-9f5b-d63f3ed920eb None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:09.229440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-66960a5b-91ee-4b59-9f5b-d63f3ed920eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:09.273464 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:09.290989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1251/2504] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:09.300998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:09.302415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:09.361890 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815debb2-f4e2-4a92-9ac2-fa1f421b1cc9 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:09.365537 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b093cf1-1d80-4aea-b716-8cd2644ca58b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:09.366140 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b093cf1-1d80-4aea-b716-8cd2644ca58b service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:09.366418 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b093cf1-1d80-4aea-b716-8cd2644ca58b service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:09.366690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b093cf1-1d80-4aea-b716-8cd2644ca58b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:09.376906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b093cf1-1d80-4aea-b716-8cd2644ca58b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:09.378368 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b093cf1-1d80-4aea-b716-8cd2644ca58b service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:09.388515 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1254/2505] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:09.737225 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66960a5b-91ee-4b59-9f5b-d63f3ed920eb None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:09.739779 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-66960a5b-91ee-4b59-9f5b-d63f3ed920eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:09.791668 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66960a5b-91ee-4b59-9f5b-d63f3ed920eb None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:09.802094 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:09.802958 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:09.803423 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:09.803866 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] PATH_INFO: `/v3/projects/bd68f72a421e464f9a1a09e8d78f610d` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:09.805223 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd68f72a421e464f9a1a09e8d78f610d)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:09.807307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:09.898381 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd68f72a421e464f9a1a09e8d78f610d', 'request_id': 'req-321d5bad-a99a-4544-b4a9-f2d54b459a20'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.906077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd68f72a421e464f9a1a09e8d78f610d', 'request_id': 'req-321d5bad-a99a-4544-b4a9-f2d54b459a20'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.913740 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd68f72a421e464f9a1a09e8d78f610d', 'request_id': 'req-321d5bad-a99a-4544-b4a9-f2d54b459a20'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.928253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] The token cache is being invalidate because project bd68f72a421e464f9a1a09e8d78f610d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:09.928634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.929453 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:09.930885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1252/2506] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:09 2022] DELETE /identity/v3/projects/bd68f72a421e464f9a1a09e8d78f610d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:09.935402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b093cf1-1d80-4aea-b716-8cd2644ca58b service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:09.946448 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b093cf1-1d80-4aea-b716-8cd2644ca58b service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:10.047530 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:10.048399 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:10.048697 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:10.049026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] PATH_INFO: `/v3/users/0c9fdd80423d4348a0df1d6ddd119c2f` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:10.052818 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9fdd80423d4348a0df1d6ddd119c2f)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:10.056287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:10.127028 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9fdd80423d4348a0df1d6ddd119c2f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.135696 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9fdd80423d4348a0df1d6ddd119c2f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.136656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9fdd80423d4348a0df1d6ddd119c2f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.140713 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9fdd80423d4348a0df1d6ddd119c2f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.150500 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-c77529d8-f094-4759-9718-6536dded35c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9fdd80423d4348a0df1d6ddd119c2f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.173706 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1255/2507] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:09 2022] DELETE /identity/v3/users/0c9fdd80423d4348a0df1d6ddd119c2f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:49:10.549440 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:10.550997 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:10.578476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d5bad-a99a-4544-b4a9-f2d54b459a20 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:10.647780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:10.648505 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:10.648788 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:10.649091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] PATH_INFO: `/v3/projects/fce42580a65b44a09c1bf06ec0036317` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:10.650313 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fce42580a65b44a09c1bf06ec0036317)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:10.652920 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:10.725570 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fce42580a65b44a09c1bf06ec0036317', 'request_id': 'req-195460d7-fffb-4e83-8748-1bf7ad41e2e1'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.732909 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fce42580a65b44a09c1bf06ec0036317', 'request_id': 'req-195460d7-fffb-4e83-8748-1bf7ad41e2e1'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.741933 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fce42580a65b44a09c1bf06ec0036317', 'request_id': 'req-195460d7-fffb-4e83-8748-1bf7ad41e2e1'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.765061 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] The token cache is being invalidate because project fce42580a65b44a09c1bf06ec0036317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:10.765493 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.766278 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-195460d7-fffb-4e83-8748-1bf7ad41e2e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:10.767803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1253/2508] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:10 2022] DELETE /identity/v3/projects/fce42580a65b44a09c1bf06ec0036317 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:10.772668 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c5fec-f18d-4b62-a613-e3a33cde7627 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:10.773056 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c5fec-f18d-4b62-a613-e3a33cde7627 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:10.773431 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c5fec-f18d-4b62-a613-e3a33cde7627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:10.789247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:10.796074 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-788c5fec-f18d-4b62-a613-e3a33cde7627 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:10.797978 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-788c5fec-f18d-4b62-a613-e3a33cde7627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:10.865189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1256/2509] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:49:10.869657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef47c48-22e2-4554-9ae3-a76e90bcc35f None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:10.873069 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef47c48-22e2-4554-9ae3-a76e90bcc35f None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:10.873253 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef47c48-22e2-4554-9ae3-a76e90bcc35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:10.881312 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:10.896010 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-8ef47c48-22e2-4554-9ae3-a76e90bcc35f None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:10.898261 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8ef47c48-22e2-4554-9ae3-a76e90bcc35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:10.910157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1254/2510] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:10.914267 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c5fec-f18d-4b62-a613-e3a33cde7627 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:10.925188 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c5fec-f18d-4b62-a613-e3a33cde7627 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:10.929936 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f014105-e90d-4018-b7ff-777375c2195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:10.930656 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f014105-e90d-4018-b7ff-777375c2195e admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:10.930998 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f014105-e90d-4018-b7ff-777375c2195e admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:10.931339 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f014105-e90d-4018-b7ff-777375c2195e admin admin] PATH_INFO: `/v3/domains` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:10.931994 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f014105-e90d-4018-b7ff-777375c2195e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:49:10.932305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f014105-e90d-4018-b7ff-777375c2195e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:10.934023 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f014105-e90d-4018-b7ff-777375c2195e admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:10.948403 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1257/2511] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:49:10.953573 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef47c48-22e2-4554-9ae3-a76e90bcc35f None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:10.964004 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef47c48-22e2-4554-9ae3-a76e90bcc35f None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:10.969533 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4b32-5bbb-48d3-b61b-b2a25854272c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:10.970157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4b32-5bbb-48d3-b61b-b2a25854272c admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:10.970438 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4b32-5bbb-48d3-b61b-b2a25854272c admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:10.970711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4b32-5bbb-48d3-b61b-b2a25854272c admin admin] PATH_INFO: `/v3/projects` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:10.971353 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4b32-5bbb-48d3-b61b-b2a25854272c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:10.973345 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4b32-5bbb-48d3-b61b-b2a25854272c admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:11.010696 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1255/2512] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:49:10 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:11.015622 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f014105-e90d-4018-b7ff-777375c2195e admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:11.026263 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f014105-e90d-4018-b7ff-777375c2195e admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:11.095192 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee2a60-c8f1-48c8-82e4-78b7759acfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:11.096100 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee2a60-c8f1-48c8-82e4-78b7759acfa8 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:11.096409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee2a60-c8f1-48c8-82e4-78b7759acfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:11.096695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee2a60-c8f1-48c8-82e4-78b7759acfa8 admin admin] PATH_INFO: `/v3/users` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:11.097336 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee2a60-c8f1-48c8-82e4-78b7759acfa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:11.099286 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee2a60-c8f1-48c8-82e4-78b7759acfa8 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:11.132297 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1258/2513] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:49:11 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:11.133787 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4b32-5bbb-48d3-b61b-b2a25854272c admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:11.144911 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4b32-5bbb-48d3-b61b-b2a25854272c admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:11.150000 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763cfd6-7e48-43c6-987f-edeb45fa83a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:11.150603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763cfd6-7e48-43c6-987f-edeb45fa83a0 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:11.151042 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763cfd6-7e48-43c6-987f-edeb45fa83a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:11.151322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763cfd6-7e48-43c6-987f-edeb45fa83a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:11.152127 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763cfd6-7e48-43c6-987f-edeb45fa83a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:11.153863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763cfd6-7e48-43c6-987f-edeb45fa83a0 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:11.164813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1256/2514] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:49:11.168888 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee2a60-c8f1-48c8-82e4-78b7759acfa8 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:11.179030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee2a60-c8f1-48c8-82e4-78b7759acfa8 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:11.184293 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e375d-0f45-4d5e-8e74-166d136fb570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:11.185075 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e375d-0f45-4d5e-8e74-166d136fb570 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:11.185376 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e375d-0f45-4d5e-8e74-166d136fb570 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:11.185650 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e375d-0f45-4d5e-8e74-166d136fb570 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:11.186814 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e375d-0f45-4d5e-8e74-166d136fb570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:11.188473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e375d-0f45-4d5e-8e74-166d136fb570 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:11.199444 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1259/2515] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:11.204315 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763cfd6-7e48-43c6-987f-edeb45fa83a0 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:11.214724 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763cfd6-7e48-43c6-987f-edeb45fa83a0 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:11.220693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43975e0d-77cb-4cff-8628-822f638e3788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:11.221364 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43975e0d-77cb-4cff-8628-822f638e3788 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:11.221642 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43975e0d-77cb-4cff-8628-822f638e3788 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:11.221907 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43975e0d-77cb-4cff-8628-822f638e3788 admin admin] PATH_INFO: `/v3/projects/ae01b02b59134de68e01864ac6fea0ec/users/be10bd6e731045beb2f94868563df95a/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:11.238634 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43975e0d-77cb-4cff-8628-822f638e3788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae01b02b59134de68e01864ac6fea0ec, user_id=be10bd6e731045beb2f94868563df95a, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:11.241022 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43975e0d-77cb-4cff-8628-822f638e3788 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:11.260072 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1257/2516] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:49:11 2022] PUT /identity/v3/projects/ae01b02b59134de68e01864ac6fea0ec/users/be10bd6e731045beb2f94868563df95a/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:11.271800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ffcaa-268e-411a-9477-95471072131e None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:11.272326 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ffcaa-268e-411a-9477-95471072131e None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:11.272963 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ffcaa-268e-411a-9477-95471072131e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:11.309447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-6a1ffcaa-268e-411a-9477-95471072131e None None] MFA Rules not processed for user `be10bd6e731045beb2f94868563df95a`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:11.312612 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6a1ffcaa-268e-411a-9477-95471072131e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:11.394402 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1260/2517] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:11.410374 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43975e0d-77cb-4cff-8628-822f638e3788 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:11.412190 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-43975e0d-77cb-4cff-8628-822f638e3788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:11.472201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43975e0d-77cb-4cff-8628-822f638e3788 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:11.475913 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8daec-9a94-46fd-b4c4-c7de90699c5d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cacfce5c9d07471cbe1713a4d6aa29f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:11.476564 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8daec-9a94-46fd-b4c4-c7de90699c5d service glance] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:11.476843 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8daec-9a94-46fd-b4c4-c7de90699c5d service glance] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:11.477383 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8daec-9a94-46fd-b4c4-c7de90699c5d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:11.487344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8daec-9a94-46fd-b4c4-c7de90699c5d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:11.488856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8daec-9a94-46fd-b4c4-c7de90699c5d service glance] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:11.499592 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1258/2518] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:21.389785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:21.495020 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:24.335807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3106508-29f1-4f2f-99c3-039a7e162704 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:24.336276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3106508-29f1-4f2f-99c3-039a7e162704 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:24.336396 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3106508-29f1-4f2f-99c3-039a7e162704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:24.355474 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-c3106508-29f1-4f2f-99c3-039a7e162704 None None] MFA Rules not processed for user `57c1b64154514e43aec44f21af65cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:24.356691 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-c3106508-29f1-4f2f-99c3-039a7e162704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:24.410878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1261/2519] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:34.403695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=25 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:37.727892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8daec-9a94-46fd-b4c4-c7de90699c5d service glance] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:37.790828 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8daec-9a94-46fd-b4c4-c7de90699c5d service glance] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:37.795293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:37.795918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:37.796196 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:37.796508 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] PATH_INFO: `/v3/users/be10bd6e731045beb2f94868563df95a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:37.797602 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be10bd6e731045beb2f94868563df95a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:37.799507 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:37.846302 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be10bd6e731045beb2f94868563df95a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:37.853885 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be10bd6e731045beb2f94868563df95a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:37.854557 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be10bd6e731045beb2f94868563df95a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:37.855173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be10bd6e731045beb2f94868563df95a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:37.862673 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be10bd6e731045beb2f94868563df95a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:37.882103 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1259/2520] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:37 2022] DELETE /identity/v3/users/be10bd6e731045beb2f94868563df95a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:37.886572 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb2b89-f0e1-41a7-baa3-3cfc67d69c4a None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:37.887025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb2b89-f0e1-41a7-baa3-3cfc67d69c4a None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:37.887359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb2b89-f0e1-41a7-baa3-3cfc67d69c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:37.906744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-deeb2b89-f0e1-41a7-baa3-3cfc67d69c4a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:37.908071 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-deeb2b89-f0e1-41a7-baa3-3cfc67d69c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:37.970644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1262/2521] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:37.980971 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:37.982621 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:38.064837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355af87-ab2b-46d7-8824-7c1fc5733e41 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:38.069276 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ead4f-1f61-4e14-8d4e-aa4aa8431770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:38.069868 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ead4f-1f61-4e14-8d4e-aa4aa8431770 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.070145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ead4f-1f61-4e14-8d4e-aa4aa8431770 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.070415 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ead4f-1f61-4e14-8d4e-aa4aa8431770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.079136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ead4f-1f61-4e14-8d4e-aa4aa8431770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:38.080615 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ead4f-1f61-4e14-8d4e-aa4aa8431770 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:38.090653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1260/2522] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:38.361287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb2b89-f0e1-41a7-baa3-3cfc67d69c4a None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:38.362734 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-deeb2b89-f0e1-41a7-baa3-3cfc67d69c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:38.377547 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb2b89-f0e1-41a7-baa3-3cfc67d69c4a None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:38.382883 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:38.383520 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.383800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.384078 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] PATH_INFO: `/v3/projects/ae01b02b59134de68e01864ac6fea0ec` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.385321 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae01b02b59134de68e01864ac6fea0ec)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:38.388477 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:38.434960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae01b02b59134de68e01864ac6fea0ec', 'request_id': 'req-399bf993-41f8-4e9c-bb02-92f98de14db6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:38.440579 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae01b02b59134de68e01864ac6fea0ec', 'request_id': 'req-399bf993-41f8-4e9c-bb02-92f98de14db6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:38.449655 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae01b02b59134de68e01864ac6fea0ec', 'request_id': 'req-399bf993-41f8-4e9c-bb02-92f98de14db6'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:38.462367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] The token cache is being invalidate because project ae01b02b59134de68e01864ac6fea0ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:38.462658 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:38.463310 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-399bf993-41f8-4e9c-bb02-92f98de14db6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:38.465361 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1263/2523] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:38 2022] DELETE /identity/v3/projects/ae01b02b59134de68e01864ac6fea0ec => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:38.470443 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e42e47-c52a-4a57-b2e2-ae383ab1849a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.470806 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e42e47-c52a-4a57-b2e2-ae383ab1849a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.471095 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e42e47-c52a-4a57-b2e2-ae383ab1849a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.479867 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:38.494071 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-84e42e47-c52a-4a57-b2e2-ae383ab1849a None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:38.495329 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-84e42e47-c52a-4a57-b2e2-ae383ab1849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:38.565355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1261/2524] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:38.568928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a20b9-248e-46ab-b152-d8ec169cd141 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.568928 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a20b9-248e-46ab-b152-d8ec169cd141 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.569359 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a20b9-248e-46ab-b152-d8ec169cd141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.602658 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-183a20b9-248e-46ab-b152-d8ec169cd141 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:38.603896 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-183a20b9-248e-46ab-b152-d8ec169cd141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:38.619324 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1264/2525] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:38.621022 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e42e47-c52a-4a57-b2e2-ae383ab1849a None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:38.630920 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e42e47-c52a-4a57-b2e2-ae383ab1849a None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:38.637230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1f51f-64ca-4f1c-a6ca-d96aa8a4554f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:38.638244 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1f51f-64ca-4f1c-a6ca-d96aa8a4554f admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.638767 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1f51f-64ca-4f1c-a6ca-d96aa8a4554f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.639236 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1f51f-64ca-4f1c-a6ca-d96aa8a4554f admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.639967 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1f51f-64ca-4f1c-a6ca-d96aa8a4554f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:49:38.640803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1f51f-64ca-4f1c-a6ca-d96aa8a4554f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:38.642049 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1f51f-64ca-4f1c-a6ca-d96aa8a4554f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:38.656382 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1262/2526] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:49:38.659934 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a20b9-248e-46ab-b152-d8ec169cd141 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:38.671304 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a20b9-248e-46ab-b152-d8ec169cd141 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:38.676207 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4c866-c32a-4196-8abd-266ef8bbc975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:38.677206 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4c866-c32a-4196-8abd-266ef8bbc975 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.677492 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4c866-c32a-4196-8abd-266ef8bbc975 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.677765 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4c866-c32a-4196-8abd-266ef8bbc975 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.678409 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4c866-c32a-4196-8abd-266ef8bbc975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:38.680312 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4c866-c32a-4196-8abd-266ef8bbc975 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:38.709714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1265/2527] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:49:38 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:38.713691 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1f51f-64ca-4f1c-a6ca-d96aa8a4554f admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:38.725703 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1f51f-64ca-4f1c-a6ca-d96aa8a4554f admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:38.785230 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d58f5-ff60-4ed4-b91d-341a1d3c1493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:38.786350 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d58f5-ff60-4ed4-b91d-341a1d3c1493 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.786637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d58f5-ff60-4ed4-b91d-341a1d3c1493 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.786916 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d58f5-ff60-4ed4-b91d-341a1d3c1493 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.787585 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d58f5-ff60-4ed4-b91d-341a1d3c1493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:38.789552 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d58f5-ff60-4ed4-b91d-341a1d3c1493 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:38.810134 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1263/2528] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:49:38 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:38.814363 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4c866-c32a-4196-8abd-266ef8bbc975 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:38.825097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4c866-c32a-4196-8abd-266ef8bbc975 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:38.829875 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd528bf-fbe9-4e6c-b927-a7452175f90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:38.830468 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd528bf-fbe9-4e6c-b927-a7452175f90d admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.830746 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd528bf-fbe9-4e6c-b927-a7452175f90d admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.831015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd528bf-fbe9-4e6c-b927-a7452175f90d admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.831647 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd528bf-fbe9-4e6c-b927-a7452175f90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:38.833395 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd528bf-fbe9-4e6c-b927-a7452175f90d admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:38.843225 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1266/2529] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:38.847269 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d58f5-ff60-4ed4-b91d-341a1d3c1493 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:38.857461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d58f5-ff60-4ed4-b91d-341a1d3c1493 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:38.861985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142933f-40b2-4d4f-ac7c-b4d92adb2300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:38.862576 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142933f-40b2-4d4f-ac7c-b4d92adb2300 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.862851 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142933f-40b2-4d4f-ac7c-b4d92adb2300 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.863122 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142933f-40b2-4d4f-ac7c-b4d92adb2300 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.864287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142933f-40b2-4d4f-ac7c-b4d92adb2300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:38.865873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142933f-40b2-4d4f-ac7c-b4d92adb2300 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:38.876086 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1264/2530] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:38.880193 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd528bf-fbe9-4e6c-b927-a7452175f90d admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:38.890559 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd528bf-fbe9-4e6c-b927-a7452175f90d admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:38.895775 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a506a69-6681-436d-b097-d14b5e9e66f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:38.896454 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a506a69-6681-436d-b097-d14b5e9e66f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.896744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a506a69-6681-436d-b097-d14b5e9e66f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.897017 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a506a69-6681-436d-b097-d14b5e9e66f8 admin admin] PATH_INFO: `/v3/projects/0d8d47f9fcae49139403ea677fe828f9/users/814de976ef3a4a8c8afa54087bd0063d/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:38.912810 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a506a69-6681-436d-b097-d14b5e9e66f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8d47f9fcae49139403ea677fe828f9, user_id=814de976ef3a4a8c8afa54087bd0063d, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:38.915494 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a506a69-6681-436d-b097-d14b5e9e66f8 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:38.929126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1267/2531] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:49:38 2022] PUT /identity/v3/projects/0d8d47f9fcae49139403ea677fe828f9/users/814de976ef3a4a8c8afa54087bd0063d/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:38.997904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a39edb-2f66-48a3-9b34-9dfedd6578e9 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:38.998110 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a39edb-2f66-48a3-9b34-9dfedd6578e9 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:38.998391 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a39edb-2f66-48a3-9b34-9dfedd6578e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:39.026402 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-c6a39edb-2f66-48a3-9b34-9dfedd6578e9 None None] MFA Rules not processed for user `814de976ef3a4a8c8afa54087bd0063d`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:39.027611 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c6a39edb-2f66-48a3-9b34-9dfedd6578e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:39.093003 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:39.099795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1265/2532] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:39.110772 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a506a69-6681-436d-b097-d14b5e9e66f8 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:39.112524 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5a506a69-6681-436d-b097-d14b5e9e66f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:39.136089 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:39.195925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a506a69-6681-436d-b097-d14b5e9e66f8 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:39.199805 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53518a09-3db9-4be9-b7af-95f3eeec6313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:39.200447 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53518a09-3db9-4be9-b7af-95f3eeec6313 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:39.200733 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53518a09-3db9-4be9-b7af-95f3eeec6313 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:39.201030 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53518a09-3db9-4be9-b7af-95f3eeec6313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:39.210389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53518a09-3db9-4be9-b7af-95f3eeec6313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:39.211819 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53518a09-3db9-4be9-b7af-95f3eeec6313 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:39.220788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1268/2533] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:47.695794 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a39edb-2f66-48a3-9b34-9dfedd6578e9 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:47.757795 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a39edb-2f66-48a3-9b34-9dfedd6578e9 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:47.762646 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:47.763272 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:47.763549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:47.763822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] PATH_INFO: `/v3/users/814de976ef3a4a8c8afa54087bd0063d` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:47.764946 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=814de976ef3a4a8c8afa54087bd0063d)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:47.766859 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:47.815591 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '814de976ef3a4a8c8afa54087bd0063d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:47.822579 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '814de976ef3a4a8c8afa54087bd0063d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:47.823264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '814de976ef3a4a8c8afa54087bd0063d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:47.823883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '814de976ef3a4a8c8afa54087bd0063d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:47.833429 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '814de976ef3a4a8c8afa54087bd0063d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:47.856976 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1266/2534] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:47 2022] DELETE /identity/v3/users/814de976ef3a4a8c8afa54087bd0063d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:47.861701 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74ae43-1867-4907-b405-3fef4786a075 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:47.862163 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74ae43-1867-4907-b405-3fef4786a075 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:47.862277 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74ae43-1867-4907-b405-3fef4786a075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:47.882700 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-1f74ae43-1867-4907-b405-3fef4786a075 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:47.883915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1f74ae43-1867-4907-b405-3fef4786a075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:47.948123 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1269/2535] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:47.958239 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:47.959614 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:48.015764 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58623e55-a055-45c9-b946-cebe6be0f924 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:48.019382 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c35a7-20c5-4a21-91bf-be7a56b03a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:48.020586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c35a7-20c5-4a21-91bf-be7a56b03a03 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.020814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c35a7-20c5-4a21-91bf-be7a56b03a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.021091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c35a7-20c5-4a21-91bf-be7a56b03a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.029877 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c35a7-20c5-4a21-91bf-be7a56b03a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:48.031299 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c35a7-20c5-4a21-91bf-be7a56b03a03 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:48.041362 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1267/2536] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:48.350925 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74ae43-1867-4907-b405-3fef4786a075 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:48.352677 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-1f74ae43-1867-4907-b405-3fef4786a075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:48.368145 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74ae43-1867-4907-b405-3fef4786a075 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:48.374033 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:48.374685 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.374968 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.375246 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] PATH_INFO: `/v3/projects/0d8d47f9fcae49139403ea677fe828f9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.376481 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8d47f9fcae49139403ea677fe828f9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:48.378394 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:48.425661 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8d47f9fcae49139403ea677fe828f9', 'request_id': 'req-db54cd73-5a10-43b5-bd81-3b47df39963f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:48.431391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8d47f9fcae49139403ea677fe828f9', 'request_id': 'req-db54cd73-5a10-43b5-bd81-3b47df39963f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:48.438744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8d47f9fcae49139403ea677fe828f9', 'request_id': 'req-db54cd73-5a10-43b5-bd81-3b47df39963f'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:48.450731 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] The token cache is being invalidate because project 0d8d47f9fcae49139403ea677fe828f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:48.451021 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:48.451644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-db54cd73-5a10-43b5-bd81-3b47df39963f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:48.453169 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1270/2537] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:48 2022] DELETE /identity/v3/projects/0d8d47f9fcae49139403ea677fe828f9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:48.458792 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498197b-e09a-488b-9a34-f59285598fd0 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.459079 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498197b-e09a-488b-9a34-f59285598fd0 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.459355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498197b-e09a-488b-9a34-f59285598fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.478931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-5498197b-e09a-488b-9a34-f59285598fd0 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:48.480167 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-5498197b-e09a-488b-9a34-f59285598fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:48.493556 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:48.541393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1268/2538] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:48.549010 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a60b7-b341-4cde-a5ef-70a805051084 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.549295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a60b7-b341-4cde-a5ef-70a805051084 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.549567 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a60b7-b341-4cde-a5ef-70a805051084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.570471 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-3c8a60b7-b341-4cde-a5ef-70a805051084 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:48.571664 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3c8a60b7-b341-4cde-a5ef-70a805051084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:48.579933 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1271/2539] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:48.583813 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498197b-e09a-488b-9a34-f59285598fd0 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:48.593502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498197b-e09a-488b-9a34-f59285598fd0 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:48.598641 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcb34b-18d8-4c7b-8317-b0e2377b7f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:48.599310 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcb34b-18d8-4c7b-8317-b0e2377b7f19 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.599589 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcb34b-18d8-4c7b-8317-b0e2377b7f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.599863 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcb34b-18d8-4c7b-8317-b0e2377b7f19 admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.600544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcb34b-18d8-4c7b-8317-b0e2377b7f19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:49:48.600856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcb34b-18d8-4c7b-8317-b0e2377b7f19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:48.602715 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcb34b-18d8-4c7b-8317-b0e2377b7f19 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:48.614742 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1269/2540] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:49:48.618992 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a60b7-b341-4cde-a5ef-70a805051084 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:48.628063 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a60b7-b341-4cde-a5ef-70a805051084 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:48.633276 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1630704-4902-4b5e-a51f-97f22f710ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:48.633861 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1630704-4902-4b5e-a51f-97f22f710ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.634136 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1630704-4902-4b5e-a51f-97f22f710ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.634408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1630704-4902-4b5e-a51f-97f22f710ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.635095 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1630704-4902-4b5e-a51f-97f22f710ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:48.637015 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1630704-4902-4b5e-a51f-97f22f710ed3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:48.657105 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:48.667774 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1272/2541] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:49:48 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:48.671948 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcb34b-18d8-4c7b-8317-b0e2377b7f19 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:48.682925 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcb34b-18d8-4c7b-8317-b0e2377b7f19 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:48.740653 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e8a6c-5746-41aa-aabe-72a35b992f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:48.741283 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e8a6c-5746-41aa-aabe-72a35b992f98 admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.741467 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e8a6c-5746-41aa-aabe-72a35b992f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.741741 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e8a6c-5746-41aa-aabe-72a35b992f98 admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.742377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e8a6c-5746-41aa-aabe-72a35b992f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:48.744286 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e8a6c-5746-41aa-aabe-72a35b992f98 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:48.766901 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1270/2542] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:49:48 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:48.771852 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1630704-4902-4b5e-a51f-97f22f710ed3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:48.781991 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1630704-4902-4b5e-a51f-97f22f710ed3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:48.787131 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2c099-f1ba-46c6-b10d-26b42cb79b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:48.787724 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2c099-f1ba-46c6-b10d-26b42cb79b93 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.788006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2c099-f1ba-46c6-b10d-26b42cb79b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.788283 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2c099-f1ba-46c6-b10d-26b42cb79b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.789155 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2c099-f1ba-46c6-b10d-26b42cb79b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:48.790743 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2c099-f1ba-46c6-b10d-26b42cb79b93 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:48.800456 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1273/2543] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 14:49:48.805436 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e8a6c-5746-41aa-aabe-72a35b992f98 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:48.815173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e8a6c-5746-41aa-aabe-72a35b992f98 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:48.819931 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7709100-ef72-44ea-85db-cc4c0edf5de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:48.820553 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7709100-ef72-44ea-85db-cc4c0edf5de7 admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.820837 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7709100-ef72-44ea-85db-cc4c0edf5de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.821115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7709100-ef72-44ea-85db-cc4c0edf5de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.821753 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7709100-ef72-44ea-85db-cc4c0edf5de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:48.823338 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7709100-ef72-44ea-85db-cc4c0edf5de7 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:48.833500 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1271/2544] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:48.837541 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2c099-f1ba-46c6-b10d-26b42cb79b93 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:48.848744 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2c099-f1ba-46c6-b10d-26b42cb79b93 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:48.853846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656c2b4-72af-4180-ae90-5fb429c375f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:48.854519 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656c2b4-72af-4180-ae90-5fb429c375f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.854800 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656c2b4-72af-4180-ae90-5fb429c375f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.855082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656c2b4-72af-4180-ae90-5fb429c375f5 admin admin] PATH_INFO: `/v3/projects/e54745bc9bd144b1ba3835f9d9df379c/users/a887f2b7649a4d7aa763f535c06993b8/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.869905 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656c2b4-72af-4180-ae90-5fb429c375f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e54745bc9bd144b1ba3835f9d9df379c, user_id=a887f2b7649a4d7aa763f535c06993b8, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:48.872692 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656c2b4-72af-4180-ae90-5fb429c375f5 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:48.887702 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1274/2545] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:49:48 2022] PUT /identity/v3/projects/e54745bc9bd144b1ba3835f9d9df379c/users/a887f2b7649a4d7aa763f535c06993b8/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:48.899809 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2c70f-dc2f-46b7-a215-ea07ae3b65e2 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:48.900304 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2c70f-dc2f-46b7-a215-ea07ae3b65e2 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:48.900678 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2c70f-dc2f-46b7-a215-ea07ae3b65e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:48.930548 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6be2c70f-dc2f-46b7-a215-ea07ae3b65e2 None None] MFA Rules not processed for user `a887f2b7649a4d7aa763f535c06993b8`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:48.931756 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6be2c70f-dc2f-46b7-a215-ea07ae3b65e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:49.013173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1272/2546] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:49.014287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656c2b4-72af-4180-ae90-5fb429c375f5 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:49.086973 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656c2b4-72af-4180-ae90-5fb429c375f5 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:49.091832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e95990-0535-47df-a368-1209ca65db8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:49.092593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e95990-0535-47df-a368-1209ca65db8b admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:49.092880 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e95990-0535-47df-a368-1209ca65db8b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:49.093151 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e95990-0535-47df-a368-1209ca65db8b admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:49.093783 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e95990-0535-47df-a368-1209ca65db8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:49.095679 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e95990-0535-47df-a368-1209ca65db8b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:49.136382 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1275/2547] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:49:49 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 14:49:49.138514 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2c70f-dc2f-46b7-a215-ea07ae3b65e2 None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:49.149486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2c70f-dc2f-46b7-a215-ea07ae3b65e2 None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:49.161729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:49.208919 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91655e57-59cd-41ea-bad7-b3ce74b5461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:49.209528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91655e57-59cd-41ea-bad7-b3ce74b5461e admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:49.209804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91655e57-59cd-41ea-bad7-b3ce74b5461e admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:49.210073 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91655e57-59cd-41ea-bad7-b3ce74b5461e admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:49.210708 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91655e57-59cd-41ea-bad7-b3ce74b5461e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:49.212658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91655e57-59cd-41ea-bad7-b3ce74b5461e admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:49.235612 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1273/2548] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:49:49 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:49:49.240560 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e95990-0535-47df-a368-1209ca65db8b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:49.250779 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e95990-0535-47df-a368-1209ca65db8b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:49.255735 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed5ab8-189a-4e7d-a008-1d179a3829af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:49.256453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed5ab8-189a-4e7d-a008-1d179a3829af admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:49.256802 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed5ab8-189a-4e7d-a008-1d179a3829af admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:49.257142 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed5ab8-189a-4e7d-a008-1d179a3829af admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:49.257988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed5ab8-189a-4e7d-a008-1d179a3829af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:49.259822 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed5ab8-189a-4e7d-a008-1d179a3829af admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:49.274299 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1276/2549] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:49.275476 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91655e57-59cd-41ea-bad7-b3ce74b5461e admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:49.285637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91655e57-59cd-41ea-bad7-b3ce74b5461e admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:49.290461 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d9eab-d08f-46ab-af0a-a250708385e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:49.291147 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d9eab-d08f-46ab-af0a-a250708385e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:49.291425 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d9eab-d08f-46ab-af0a-a250708385e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:49.291692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d9eab-d08f-46ab-af0a-a250708385e2 admin admin] PATH_INFO: `/v3/projects/121b6fca30894a68a0be72e3728ee10d/users/49734f30ed4c429693f6de066313b406/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:49.308485 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d9eab-d08f-46ab-af0a-a250708385e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121b6fca30894a68a0be72e3728ee10d, user_id=49734f30ed4c429693f6de066313b406, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:49.311292 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d9eab-d08f-46ab-af0a-a250708385e2 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:49.327439 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1274/2550] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:49:49 2022] PUT /identity/v3/projects/121b6fca30894a68a0be72e3728ee10d/users/49734f30ed4c429693f6de066313b406/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:49.331400 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed5ab8-189a-4e7d-a008-1d179a3829af admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:49.342808 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed5ab8-189a-4e7d-a008-1d179a3829af admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:49.410929 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8174e08-4f46-43b7-9518-065cc116e4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:49.411550 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8174e08-4f46-43b7-9518-065cc116e4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:49.411837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8174e08-4f46-43b7-9518-065cc116e4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:49.412118 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8174e08-4f46-43b7-9518-065cc116e4a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:49.412832 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8174e08-4f46-43b7-9518-065cc116e4a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:49.414437 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8174e08-4f46-43b7-9518-065cc116e4a1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:49.430399 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d9eab-d08f-46ab-af0a-a250708385e2 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:49.437466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1277/2551] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:49:49.443265 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d9eab-d08f-46ab-af0a-a250708385e2 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:49.448804 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716260dc-c993-4fd4-ac5c-3183163d9266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:49.449597 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716260dc-c993-4fd4-ac5c-3183163d9266 admin admin] REQUEST_METHOD: `PUT` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:49.449949 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716260dc-c993-4fd4-ac5c-3183163d9266 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:49.450322 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716260dc-c993-4fd4-ac5c-3183163d9266 admin admin] PATH_INFO: `/v3/projects/121b6fca30894a68a0be72e3728ee10d/users/49734f30ed4c429693f6de066313b406/roles/d8edc38e326243f28c9b81911ac5ed8f` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:49.452486 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716260dc-c993-4fd4-ac5c-3183163d9266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121b6fca30894a68a0be72e3728ee10d, user_id=49734f30ed4c429693f6de066313b406, role_id=d8edc38e326243f28c9b81911ac5ed8f)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:49.455237 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716260dc-c993-4fd4-ac5c-3183163d9266 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:49.468297 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1275/2552] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:49:49 2022] PUT /identity/v3/projects/121b6fca30894a68a0be72e3728ee10d/users/49734f30ed4c429693f6de066313b406/roles/d8edc38e326243f28c9b81911ac5ed8f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:49.481985 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bb712-e90f-40b7-8c40-034293c4e2be None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:49.482366 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bb712-e90f-40b7-8c40-034293c4e2be None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:49.482739 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bb712-e90f-40b7-8c40-034293c4e2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:49.511866 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-d07bb712-e90f-40b7-8c40-034293c4e2be None None] MFA Rules not processed for user `49734f30ed4c429693f6de066313b406`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:49.513212 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-d07bb712-e90f-40b7-8c40-034293c4e2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:49.533988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:49.589895 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1278/2553] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:49.602610 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716260dc-c993-4fd4-ac5c-3183163d9266 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:49.604790 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-716260dc-c993-4fd4-ac5c-3183163d9266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:49.677058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716260dc-c993-4fd4-ac5c-3183163d9266 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:49.680654 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cb28c-ae7e-4b55-816b-c967e8918061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:49.681273 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cb28c-ae7e-4b55-816b-c967e8918061 service cinder] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:49.681568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cb28c-ae7e-4b55-816b-c967e8918061 service cinder] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:49.681797 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cb28c-ae7e-4b55-816b-c967e8918061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:49.749817 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cb28c-ae7e-4b55-816b-c967e8918061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:49.751431 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cb28c-ae7e-4b55-816b-c967e8918061 service cinder] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:49.766119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1276/2554] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:49:58.667843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bb712-e90f-40b7-8c40-034293c4e2be None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:58.722008 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7c317-f04f-4efd-b6b1-d7d460c2fe1a None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:58.722462 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7c317-f04f-4efd-b6b1-d7d460c2fe1a None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:58.722947 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7c317-f04f-4efd-b6b1-d7d460c2fe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:58.749882 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-6bd7c317-f04f-4efd-b6b1-d7d460c2fe1a None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:58.751098 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-6bd7c317-f04f-4efd-b6b1-d7d460c2fe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:58.757230 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bb712-e90f-40b7-8c40-034293c4e2be None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:58.762101 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:58.762723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:58.763000 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:58.763269 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] PATH_INFO: `/v3/users/a887f2b7649a4d7aa763f535c06993b8` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:58.764565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a887f2b7649a4d7aa763f535c06993b8)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:58.766678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:58.769282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:58.821377 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a887f2b7649a4d7aa763f535c06993b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:58.821521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1277/2555] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Oct 20 14:49:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 14:49:58.831291 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a887f2b7649a4d7aa763f535c06993b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:58.832058 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a887f2b7649a4d7aa763f535c06993b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:58.832703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a887f2b7649a4d7aa763f535c06993b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:58.843333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a887f2b7649a4d7aa763f535c06993b8'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:58.864657 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1279/2556] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:58 2022] DELETE /identity/v3/users/a887f2b7649a4d7aa763f535c06993b8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:58.869125 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac09f-3f2e-498f-9d46-dfda64cc64ae None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:58.869344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac09f-3f2e-498f-9d46-dfda64cc64ae None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:58.869853 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac09f-3f2e-498f-9d46-dfda64cc64ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:58.880446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:58.883150 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:58.892117 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-8c5ac09f-3f2e-498f-9d46-dfda64cc64ae None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:49:58.893346 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8c5ac09f-3f2e-498f-9d46-dfda64cc64ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:58.928165 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:58.959001 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b5751-e41e-4fd8-8339-483c77ea4252 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:58.960521 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1278/2557] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:49:58.964725 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:58.965305 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:58.965583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:58.965856 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:58.967648 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:58.970417 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac09f-3f2e-498f-9d46-dfda64cc64ae None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:58.971798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-8c5ac09f-3f2e-498f-9d46-dfda64cc64ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:59.033618 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:59.034243 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac09f-3f2e-498f-9d46-dfda64cc64ae None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:59.035330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:59.038532 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd5f4d-c542-46b8-9a7a-a49953d71c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:59.039140 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd5f4d-c542-46b8-9a7a-a49953d71c15 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:59.039434 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd5f4d-c542-46b8-9a7a-a49953d71c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:59.039698 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd5f4d-c542-46b8-9a7a-a49953d71c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:59.043974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1280/2558] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:49:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 14:49:59.049586 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd5f4d-c542-46b8-9a7a-a49953d71c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:59.051177 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd5f4d-c542-46b8-9a7a-a49953d71c15 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:59.061796 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1279/2559] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:49:59.377380 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:59.378932 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:49:59.408135 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ea366-6998-4b29-9af0-2fa952225109 service swift] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:59.416670 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:59.417562 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:59.417851 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:59.418287 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] PATH_INFO: `/v3/projects/e54745bc9bd144b1ba3835f9d9df379c` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:59.420035 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e54745bc9bd144b1ba3835f9d9df379c)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:59.422718 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:59.499115 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e54745bc9bd144b1ba3835f9d9df379c', 'request_id': 'req-341568d0-392e-4a4b-8c52-95dbaea657a2'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.507073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e54745bc9bd144b1ba3835f9d9df379c', 'request_id': 'req-341568d0-392e-4a4b-8c52-95dbaea657a2'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.518164 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e54745bc9bd144b1ba3835f9d9df379c', 'request_id': 'req-341568d0-392e-4a4b-8c52-95dbaea657a2'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.533640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] The token cache is being invalidate because project e54745bc9bd144b1ba3835f9d9df379c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:49:59.534097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.534852 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.537213 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1281/2560] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:49:59 2022] DELETE /identity/v3/projects/e54745bc9bd144b1ba3835f9d9df379c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:49:59.542282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd5f4d-c542-46b8-9a7a-a49953d71c15 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:49:59.553861 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd5f4d-c542-46b8-9a7a-a49953d71c15 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:49:59.630904 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:49:59.633126 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:49:59.633857 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:49:59.634345 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] PATH_INFO: `/v3/users/49734f30ed4c429693f6de066313b406` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:49:59.636080 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] RBAC: Authorizing `identity:delete_user(user_id=49734f30ed4c429693f6de066313b406)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:49:59.638923 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:49:59.657135 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:49:59.708544 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49734f30ed4c429693f6de066313b406'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.719502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49734f30ed4c429693f6de066313b406'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.720693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49734f30ed4c429693f6de066313b406'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.721693 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49734f30ed4c429693f6de066313b406'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.732320 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-fca35df8-b0b0-4f26-a7b5-12c3ab76823a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49734f30ed4c429693f6de066313b406'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:49:59.754568 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1280/2561] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:49:59 2022] DELETE /identity/v3/users/49734f30ed4c429693f6de066313b406 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:50:00.099640 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:00.101982 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:00.132837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341568d0-392e-4a4b-8c52-95dbaea657a2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:00.258006 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:00.258797 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:00.259138 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:00.259415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] PATH_INFO: `/v3/projects/121b6fca30894a68a0be72e3728ee10d` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:00.260563 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=121b6fca30894a68a0be72e3728ee10d)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:00.262531 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:00.282268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:50:00.348764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121b6fca30894a68a0be72e3728ee10d', 'request_id': 'req-eda2c9fd-5749-4f8a-8c21-88f303feafb1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:00.359874 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121b6fca30894a68a0be72e3728ee10d', 'request_id': 'req-eda2c9fd-5749-4f8a-8c21-88f303feafb1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:00.370205 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121b6fca30894a68a0be72e3728ee10d', 'request_id': 'req-eda2c9fd-5749-4f8a-8c21-88f303feafb1'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:00.389268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] The token cache is being invalidate because project 121b6fca30894a68a0be72e3728ee10d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:50:00.389947 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:00.390858 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-eda2c9fd-5749-4f8a-8c21-88f303feafb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:00.393070 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1282/2562] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:50:00 2022] DELETE /identity/v3/projects/121b6fca30894a68a0be72e3728ee10d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:50:00.398030 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c568e-6e97-4ab7-9a6c-90dc6116d91b None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:00.398394 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c568e-6e97-4ab7-9a6c-90dc6116d91b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:00.398747 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c568e-6e97-4ab7-9a6c-90dc6116d91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:00.422248 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-410c568e-6e97-4ab7-9a6c-90dc6116d91b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:50:00.423637 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-410c568e-6e97-4ab7-9a6c-90dc6116d91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:00.509730 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1281/2563] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:50:00.517367 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4a6a3-6acc-4690-9633-0d4668e0fd78 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:00.518430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4a6a3-6acc-4690-9633-0d4668e0fd78 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:00.518902 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4a6a3-6acc-4690-9633-0d4668e0fd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:00.543082 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-5ce4a6a3-6acc-4690-9633-0d4668e0fd78 None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:50:00.544610 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-5ce4a6a3-6acc-4690-9633-0d4668e0fd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:00.553735 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1283/2564] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:50:00.559390 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c568e-6e97-4ab7-9a6c-90dc6116d91b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:00.575154 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c568e-6e97-4ab7-9a6c-90dc6116d91b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:00.581121 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514953e-2a65-4d4a-86a4-9360d6a9ec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:00.581873 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514953e-2a65-4d4a-86a4-9360d6a9ec0f admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:00.582472 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514953e-2a65-4d4a-86a4-9360d6a9ec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:00.582601 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514953e-2a65-4d4a-86a4-9360d6a9ec0f admin admin] PATH_INFO: `/v3/domains` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:00.583428 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514953e-2a65-4d4a-86a4-9360d6a9ec0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82085) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 14:50:00.583739 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514953e-2a65-4d4a-86a4-9360d6a9ec0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:00.587282 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514953e-2a65-4d4a-86a4-9360d6a9ec0f admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:00.608393 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1282/2565] 10.222.0.13 () {58 vars in 1153 bytes} [Thu Oct 20 14:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 14:50:00.611208 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4a6a3-6acc-4690-9633-0d4668e0fd78 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:00.621433 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4a6a3-6acc-4690-9633-0d4668e0fd78 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:00.626690 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae255-ccc3-4c5a-bc66-4f25345ad757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:00.627302 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae255-ccc3-4c5a-bc66-4f25345ad757 admin admin] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:00.627583 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae255-ccc3-4c5a-bc66-4f25345ad757 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:00.627853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae255-ccc3-4c5a-bc66-4f25345ad757 admin admin] PATH_INFO: `/v3/projects` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:00.628521 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae255-ccc3-4c5a-bc66-4f25345ad757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:00.630430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae255-ccc3-4c5a-bc66-4f25345ad757 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:00.666633 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1284/2566] 10.222.0.13 () {60 vars in 1153 bytes} [Thu Oct 20 14:50:00 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:50:00.668694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514953e-2a65-4d4a-86a4-9360d6a9ec0f admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:00.688528 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514953e-2a65-4d4a-86a4-9360d6a9ec0f admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:00.805115 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffad68-2d32-48fd-94b2-e330be0ab44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:00.805985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffad68-2d32-48fd-94b2-e330be0ab44d admin admin] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:00.805985 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffad68-2d32-48fd-94b2-e330be0ab44d admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:00.806284 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffad68-2d32-48fd-94b2-e330be0ab44d admin admin] PATH_INFO: `/v3/users` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:00.806962 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffad68-2d32-48fd-94b2-e330be0ab44d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:00.809198 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffad68-2d32-48fd-94b2-e330be0ab44d admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:00.835534 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1283/2567] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Oct 20 14:50:00 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 14:50:00.839962 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae255-ccc3-4c5a-bc66-4f25345ad757 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:00.852482 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae255-ccc3-4c5a-bc66-4f25345ad757 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:00.858453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fe2b4-45c7-4048-9520-95f7a40eecda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:00.859548 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fe2b4-45c7-4048-9520-95f7a40eecda admin admin] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:00.860007 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fe2b4-45c7-4048-9520-95f7a40eecda admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:00.860472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fe2b4-45c7-4048-9520-95f7a40eecda admin admin] PATH_INFO: `/v3/roles` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:00.861566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fe2b4-45c7-4048-9520-95f7a40eecda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:00.863517 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fe2b4-45c7-4048-9520-95f7a40eecda admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:00.878325 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1285/2568] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:50:00.882924 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffad68-2d32-48fd-94b2-e330be0ab44d admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:00.896047 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffad68-2d32-48fd-94b2-e330be0ab44d admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:00.902745 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef56400-206b-43d1-af6a-2ceadb3ce17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:00.904034 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef56400-206b-43d1-af6a-2ceadb3ce17b admin admin] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:00.904629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef56400-206b-43d1-af6a-2ceadb3ce17b admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:00.904872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef56400-206b-43d1-af6a-2ceadb3ce17b admin admin] PATH_INFO: `/v3/roles` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:00.905596 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef56400-206b-43d1-af6a-2ceadb3ce17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:00.907143 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef56400-206b-43d1-af6a-2ceadb3ce17b admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:00.908803 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:50:00.924515 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1284/2569] 10.222.0.13 () {58 vars in 1122 bytes} [Thu Oct 20 14:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 14:50:00.926624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fe2b4-45c7-4048-9520-95f7a40eecda admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:00.938466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fe2b4-45c7-4048-9520-95f7a40eecda admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:00.946807 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3b02f-c395-407a-b34d-b48aa51073e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:00.947843 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3b02f-c395-407a-b34d-b48aa51073e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:00.948296 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3b02f-c395-407a-b34d-b48aa51073e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:00.948778 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3b02f-c395-407a-b34d-b48aa51073e4 admin admin] PATH_INFO: `/v3/projects/bdf6ecc2b1204589b802bdc83e01ceb2/users/80b69f3590fd4435affcdb5f591bdcc3/roles/a73215dcd05645c89a2f742200bbb0c9` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:00.969120 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3b02f-c395-407a-b34d-b48aa51073e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf6ecc2b1204589b802bdc83e01ceb2, user_id=80b69f3590fd4435affcdb5f591bdcc3, role_id=a73215dcd05645c89a2f742200bbb0c9)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:00.977289 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3b02f-c395-407a-b34d-b48aa51073e4 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:00.995941 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1286/2570] 10.222.0.13 () {60 vars in 1483 bytes} [Thu Oct 20 14:50:00 2022] PUT /identity/v3/projects/bdf6ecc2b1204589b802bdc83e01ceb2/users/80b69f3590fd4435affcdb5f591bdcc3/roles/a73215dcd05645c89a2f742200bbb0c9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:50:01.009680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15414b4-a11a-45f3-afe3-c8bc4a79df23 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:01.010026 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15414b4-a11a-45f3-afe3-c8bc4a79df23 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:01.010306 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15414b4-a11a-45f3-afe3-c8bc4a79df23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:01.041603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-e15414b4-a11a-45f3-afe3-c8bc4a79df23 None None] MFA Rules not processed for user `80b69f3590fd4435affcdb5f591bdcc3`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:50:01.042729 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-e15414b4-a11a-45f3-afe3-c8bc4a79df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:01.114921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1285/2571] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:50:01.127061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3b02f-c395-407a-b34d-b48aa51073e4 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:01.128915 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-58e3b02f-c395-407a-b34d-b48aa51073e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:01.196323 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3b02f-c395-407a-b34d-b48aa51073e4 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:01.200122 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1e968-6eeb-4d18-90b0-7e97535f0fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70742ba8f3f6461f8f578b323afea7cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:01.200782 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1e968-6eeb-4d18-90b0-7e97535f0fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:01.201052 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1e968-6eeb-4d18-90b0-7e97535f0fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:01.201330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1e968-6eeb-4d18-90b0-7e97535f0fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:01.210667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1e968-6eeb-4d18-90b0-7e97535f0fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:01.212113 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1e968-6eeb-4d18-90b0-7e97535f0fc5 service cinder] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:01.221995 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1287/2572] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:50:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:50:04.402405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7417a-82f9-4c91-bb03-b3f0ce8b9938 None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:04.403193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7417a-82f9-4c91-bb03-b3f0ce8b9938 None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:04.403193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7417a-82f9-4c91-bb03-b3f0ce8b9938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:04.422345 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-51a7417a-82f9-4c91-bb03-b3f0ce8b9938 None None] MFA Rules not processed for user `36a281f7f9c8435c8ad65ae4b0bf141a`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:50:04.423527 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-51a7417a-82f9-4c91-bb03-b3f0ce8b9938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:04.482928 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1286/2573] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:50:10.815306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1e968-6eeb-4d18-90b0-7e97535f0fc5 service cinder] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:10.872173 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:50:10.917408 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1e968-6eeb-4d18-90b0-7e97535f0fc5 service cinder] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:10.922593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:10.923232 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:10.923546 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:10.923853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] PATH_INFO: `/v3/users/80b69f3590fd4435affcdb5f591bdcc3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:10.925162 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b69f3590fd4435affcdb5f591bdcc3)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:10.927996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:10.997211 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b69f3590fd4435affcdb5f591bdcc3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:11.006719 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b69f3590fd4435affcdb5f591bdcc3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:11.009251 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b69f3590fd4435affcdb5f591bdcc3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:11.010178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b69f3590fd4435affcdb5f591bdcc3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:11.020128 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b69f3590fd4435affcdb5f591bdcc3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:11.044503 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1288/2574] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:50:10 2022] DELETE /identity/v3/users/80b69f3590fd4435affcdb5f591bdcc3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:50:11.048686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ffa59-cf44-46ff-b753-84ec8eeec26b None None] REQUEST_METHOD: `POST` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:11.049658 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ffa59-cf44-46ff-b753-84ec8eeec26b None None] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:11.050538 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ffa59-cf44-46ff-b753-84ec8eeec26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:11.091680 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.auth.core [None req-337ffa59-cf44-46ff-b753-84ec8eeec26b None None] MFA Rules not processed for user `75ee7ae2d2194fbf9ae79851bb412a22`. Rule list: `[]` (Enabled: `True`). {{(pid=82085) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:50:11.093406 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-337ffa59-cf44-46ff-b753-84ec8eeec26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:11.201921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1287/2575] 10.222.0.13 () {58 vars in 958 bytes} [Thu Oct 20 14:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:50:11.230065 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:11.232405 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:11.328109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55fe90-f9d3-4ff3-a62d-3f4c9fe88db0 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:11.351676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2df27-a174-4d72-8647-249a5293855d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:11.351676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2df27-a174-4d72-8647-249a5293855d service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:11.351676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2df27-a174-4d72-8647-249a5293855d service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:11.351676 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2df27-a174-4d72-8647-249a5293855d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:11.361988 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2df27-a174-4d72-8647-249a5293855d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:11.363844 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2df27-a174-4d72-8647-249a5293855d service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:11.376256 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1289/2576] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:50:12.058242 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ffa59-cf44-46ff-b753-84ec8eeec26b None None] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:12.059722 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-337ffa59-cf44-46ff-b753-84ec8eeec26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:12.077326 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ffa59-cf44-46ff-b753-84ec8eeec26b None None] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:12.104152 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:12.106765 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:12.107233 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:12.107667 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] PATH_INFO: `/v3/projects/bdf6ecc2b1204589b802bdc83e01ceb2` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:12.109136 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf6ecc2b1204589b802bdc83e01ceb2)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:12.111293 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:12.166914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:50:12.187808 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf6ecc2b1204589b802bdc83e01ceb2', 'request_id': 'req-b794714e-0a27-48c6-9779-12b09bf673f6'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:12.194802 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf6ecc2b1204589b802bdc83e01ceb2', 'request_id': 'req-b794714e-0a27-48c6-9779-12b09bf673f6'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:12.204814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf6ecc2b1204589b802bdc83e01ceb2', 'request_id': 'req-b794714e-0a27-48c6-9779-12b09bf673f6'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:12.218037 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] The token cache is being invalidate because project bdf6ecc2b1204589b802bdc83e01ceb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:50:12.218457 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:12.219196 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:12.220824 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1288/2577] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:50:12 2022] DELETE /identity/v3/projects/bdf6ecc2b1204589b802bdc83e01ceb2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:50:17.048510 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be3710-ad68-4be1-b86c-eeac018667b0 None None] REQUEST_METHOD: `POST` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:17.048972 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be3710-ad68-4be1-b86c-eeac018667b0 None None] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:17.049069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be3710-ad68-4be1-b86c-eeac018667b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:17.069257 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.auth.core [None req-57be3710-ad68-4be1-b86c-eeac018667b0 None None] MFA Rules not processed for user `04f97b428a6347e39b9b1f97ace1e93b`. Rule list: `[]` (Enabled: `True`). {{(pid=82084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 14:50:17.070504 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-57be3710-ad68-4be1-b86c-eeac018667b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:17.117061 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:50:17.126473 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1290/2578] 10.222.0.13 () {58 vars in 967 bytes} [Thu Oct 20 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 14:50:17.139407 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:17.140835 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:17.208682 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794714e-0a27-48c6-9779-12b09bf673f6 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:17.214201 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21976d55-f067-4aa7-98e2-ace2fee30a79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61d6c1cd010d4fa387fa64864419bc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:17.214798 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21976d55-f067-4aa7-98e2-ace2fee30a79 service swift] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:17.215076 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21976d55-f067-4aa7-98e2-ace2fee30a79 service swift] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:17.215349 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21976d55-f067-4aa7-98e2-ace2fee30a79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:17.223145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21976d55-f067-4aa7-98e2-ace2fee30a79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:17.225254 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21976d55-f067-4aa7-98e2-ace2fee30a79 service swift] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:17.233281 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1289/2579] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Oct 20 14:50:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 14:50:19.590873 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be3710-ad68-4be1-b86c-eeac018667b0 None None] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:19.592295 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-57be3710-ad68-4be1-b86c-eeac018667b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:19.660533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be3710-ad68-4be1-b86c-eeac018667b0 None None] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:19.665461 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:19.666079 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:19.666357 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:19.666653 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] PATH_INFO: `/v3/users/57c1b64154514e43aec44f21af65cff7` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:19.667769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c1b64154514e43aec44f21af65cff7)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:19.669706 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:19.714769 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c1b64154514e43aec44f21af65cff7'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:19.721815 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c1b64154514e43aec44f21af65cff7'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:19.722490 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c1b64154514e43aec44f21af65cff7'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:19.723077 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c1b64154514e43aec44f21af65cff7'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:19.732157 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c1b64154514e43aec44f21af65cff7'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:19.749069 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1291/2580] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:50:19 2022] DELETE /identity/v3/users/57c1b64154514e43aec44f21af65cff7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:50:19.919921 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21976d55-f067-4aa7-98e2-ace2fee30a79 service swift] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:19.932918 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21976d55-f067-4aa7-98e2-ace2fee30a79 service swift] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:19.995883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:19.996547 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:19.996822 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:19.997091 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] PATH_INFO: `/v3/projects/7bafa4c5aa6943f9a860f577c957f33d` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:19.998196 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bafa4c5aa6943f9a860f577c957f33d)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:20.000087 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:20.045006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:50:20.048123 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bafa4c5aa6943f9a860f577c957f33d', 'request_id': 'req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:20.061375 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bafa4c5aa6943f9a860f577c957f33d', 'request_id': 'req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:20.070624 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bafa4c5aa6943f9a860f577c957f33d', 'request_id': 'req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:20.081793 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] The token cache is being invalidate because project 7bafa4c5aa6943f9a860f577c957f33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:50:20.082081 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:20.082694 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:50:20.084217 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1290/2581] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:50:19 2022] DELETE /identity/v3/projects/7bafa4c5aa6943f9a860f577c957f33d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:50:22.592654 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:22.594091 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:22.655373 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0a524-b1c8-4474-af91-2affbb729639 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:22.687794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:50:22.710493 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:22.711231 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:22.711348 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:22.711621 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:22.713371 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:22.722846 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:22.725221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:22.733720 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1292/2582] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:50:22.804433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:22.805914 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:22.818636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b0e1-f6dd-4e5d-8db4-a66365a4ec47 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:22.822193 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacd0f6-a731-4c11-9909-2ba42ba43b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:22.822783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacd0f6-a731-4c11-9909-2ba42ba43b55 service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:22.823058 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacd0f6-a731-4c11-9909-2ba42ba43b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:22.823328 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacd0f6-a731-4c11-9909-2ba42ba43b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:22.825005 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0aacd0f6-a731-4c11-9909-2ba42ba43b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:22.906763 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacd0f6-a731-4c11-9909-2ba42ba43b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:22.908197 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacd0f6-a731-4c11-9909-2ba42ba43b55 service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:22.929165 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1291/2583] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:50:32.729853 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=11 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:50:32.925629 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:DELETE=4,keystone:SELECT=50,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:50:50.087590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:50.089598 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:50.103764 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef601d-9211-4ad5-989a-6f9ede739ac3 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:50.109109 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe006ce-45dc-4fd1-a084-a840bbc40977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:50.109715 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe006ce-45dc-4fd1-a084-a840bbc40977 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:50.109996 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe006ce-45dc-4fd1-a084-a840bbc40977 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:50.110272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe006ce-45dc-4fd1-a084-a840bbc40977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:50.112234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-cfe006ce-45dc-4fd1-a084-a840bbc40977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:50.126372 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe006ce-45dc-4fd1-a084-a840bbc40977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:50.127789 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe006ce-45dc-4fd1-a084-a840bbc40977 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:50.139221 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1293/2584] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:50:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:50:51.322609 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacd0f6-a731-4c11-9909-2ba42ba43b55 service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:50:51.331246 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacd0f6-a731-4c11-9909-2ba42ba43b55 service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:50:51.335865 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:50:51.337124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:50:51.337124 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:50:51.337377 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:50:51.339118 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:50:51.354379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:50:51.355989 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:50:51.368148 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1292/2585] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:50:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 14:51:00.134345 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=6 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:51:01.362856 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=6 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:51:05.530097 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe006ce-45dc-4fd1-a084-a840bbc40977 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:05.538785 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe006ce-45dc-4fd1-a084-a840bbc40977 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:05.542430 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bd274-7fbf-4828-b86e-0b89336aa750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:05.543026 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bd274-7fbf-4828-b86e-0b89336aa750 service neutron] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:05.543333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bd274-7fbf-4828-b86e-0b89336aa750 service neutron] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:05.543609 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bd274-7fbf-4828-b86e-0b89336aa750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:05.545343 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e47bd274-7fbf-4828-b86e-0b89336aa750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:05.613854 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bd274-7fbf-4828-b86e-0b89336aa750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:05.615282 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bd274-7fbf-4828-b86e-0b89336aa750 service neutron] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:05.626445 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1294/2586] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:51:06.473307 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:06.476185 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:06.489603 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c93c4-7f19-4222-bf8a-9b3d0d689d0a service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:06.495057 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:06.495689 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:06.495966 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:06.496674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] PATH_INFO: `/v3/users/36a281f7f9c8435c8ad65ae4b0bf141a` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:06.497446 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36a281f7f9c8435c8ad65ae4b0bf141a)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:06.499370 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:06.549043 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36a281f7f9c8435c8ad65ae4b0bf141a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.556405 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36a281f7f9c8435c8ad65ae4b0bf141a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.557077 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36a281f7f9c8435c8ad65ae4b0bf141a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.557674 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36a281f7f9c8435c8ad65ae4b0bf141a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.566537 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36a281f7f9c8435c8ad65ae4b0bf141a'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.585807 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1293/2587] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:51:06 2022] DELETE /identity/v3/users/36a281f7f9c8435c8ad65ae4b0bf141a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:51:06.771949 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bd274-7fbf-4828-b86e-0b89336aa750 service neutron] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:06.794126 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bd274-7fbf-4828-b86e-0b89336aa750 service neutron] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:06.869368 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:06.870025 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:06.870307 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:06.870581 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] PATH_INFO: `/v3/projects/34676c921ba04710b14cfa94eab069d3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:06.871723 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34676c921ba04710b14cfa94eab069d3)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:06.873749 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:06.939271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34676c921ba04710b14cfa94eab069d3', 'request_id': 'req-26674224-b892-42a4-809f-9a0fe75a3be3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.947421 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34676c921ba04710b14cfa94eab069d3', 'request_id': 'req-26674224-b892-42a4-809f-9a0fe75a3be3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.956630 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34676c921ba04710b14cfa94eab069d3', 'request_id': 'req-26674224-b892-42a4-809f-9a0fe75a3be3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.975542 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] The token cache is being invalidate because project 34676c921ba04710b14cfa94eab069d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:51:06.975835 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.976539 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:06.984536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1295/2588] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:51:06 2022] DELETE /identity/v3/projects/34676c921ba04710b14cfa94eab069d3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:51:12.407410 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:12.409036 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:12.423104 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3d8d5-11b5-4347-8d8d-dd257183d1a4 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:12.479841 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:12.480535 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:12.480814 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:12.481088 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] PATH_INFO: `/v3/users/ac5921418d40446c80892b04a3eee12d` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:12.482199 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac5921418d40446c80892b04a3eee12d)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:12.484112 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:12.530144 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac5921418d40446c80892b04a3eee12d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.537636 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac5921418d40446c80892b04a3eee12d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.538397 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac5921418d40446c80892b04a3eee12d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.539006 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac5921418d40446c80892b04a3eee12d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.546501 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac5921418d40446c80892b04a3eee12d'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.554335 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:51:12.565757 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1294/2589] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:51:12 2022] DELETE /identity/v3/users/ac5921418d40446c80892b04a3eee12d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:51:12.752389 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:12.753968 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:12.774906 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26674224-b892-42a4-809f-9a0fe75a3be3 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:12.796466 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:51:12.835245 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:12.835870 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:12.836152 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:12.836472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] PATH_INFO: `/v3/projects/d5614088125349f1975cc55b25dd9dbc` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:12.837590 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5614088125349f1975cc55b25dd9dbc)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:12.839485 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:12.885313 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5614088125349f1975cc55b25dd9dbc', 'request_id': 'req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.891003 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5614088125349f1975cc55b25dd9dbc', 'request_id': 'req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.898565 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5614088125349f1975cc55b25dd9dbc', 'request_id': 'req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.909401 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] The token cache is being invalidate because project d5614088125349f1975cc55b25dd9dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:51:12.909624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.910248 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:12.911714 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1296/2590] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:51:12 2022] DELETE /identity/v3/projects/d5614088125349f1975cc55b25dd9dbc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:51:12.915886 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:12.917372 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:12.930355 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e763ebd-eacd-4fc8-ae32-b5b3a7cc2cfb admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:12.987264 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:12.987892 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:12.988173 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:12.988479 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] PATH_INFO: `/v3/users/cc1a3fb9270a435b819d3a05cf1ed153` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:12.989675 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc1a3fb9270a435b819d3a05cf1ed153)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:12.991762 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:13.036523 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc1a3fb9270a435b819d3a05cf1ed153'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.043426 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc1a3fb9270a435b819d3a05cf1ed153'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.044073 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc1a3fb9270a435b819d3a05cf1ed153'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.044692 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc1a3fb9270a435b819d3a05cf1ed153'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.051717 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc1a3fb9270a435b819d3a05cf1ed153'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.069232 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1295/2591] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:51:12 2022] DELETE /identity/v3/users/cc1a3fb9270a435b819d3a05cf1ed153 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:51:13.240695 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:13.242189 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:13.257201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9f9e6-5ee2-463b-95e8-b80410e4480b admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:13.313268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:13.313896 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:13.314178 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:13.314446 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] PATH_INFO: `/v3/projects/2a583792afbd4bf2b016df8fe868fc82` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:13.315533 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a583792afbd4bf2b016df8fe868fc82)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:13.317593 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:13.366278 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a583792afbd4bf2b016df8fe868fc82', 'request_id': 'req-3ec6441d-4c63-4e0b-999b-256e01583c69'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.370829 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:51:13.372198 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a583792afbd4bf2b016df8fe868fc82', 'request_id': 'req-3ec6441d-4c63-4e0b-999b-256e01583c69'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.380568 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a583792afbd4bf2b016df8fe868fc82', 'request_id': 'req-3ec6441d-4c63-4e0b-999b-256e01583c69'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.392039 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] The token cache is being invalidate because project 2a583792afbd4bf2b016df8fe868fc82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:51:13.392330 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.392989 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:51:13.394470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1297/2592] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:51:13 2022] DELETE /identity/v3/projects/2a583792afbd4bf2b016df8fe868fc82 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 14:51:23.065379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=56,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:51:23.389031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=5,keystone:INSERT=1,keystone:DELETE=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:51:57.458842 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:57.460330 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:57.517736 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e6386-bf98-4830-84ba-905edc3829fe admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:57.563707 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:57.564308 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:57.564618 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:57.564890 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:57.566605 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:57.625149 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:57.626511 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:57.635433 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1296/2593] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:51:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:51:58.450711 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:58.452703 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:58.546391 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6441d-4c63-4e0b-999b-256e01583c69 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:58.551960 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:58.552608 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:58.552893 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:58.553167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:58.555073 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:58.570114 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:58.571667 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:58.584294 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1298/2594] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:51:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:51:59.627311 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:51:59.629055 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:59.640100 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a0a36-e45c-452a-86d2-2cb3bf8bdd2b service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:51:59.643711 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a1c29-1637-40b6-95e5-b6064206c3ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:51:59.644301 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a1c29-1637-40b6-95e5-b6064206c3ea service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:51:59.644613 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a1c29-1637-40b6-95e5-b6064206c3ea service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:51:59.644883 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a1c29-1637-40b6-95e5-b6064206c3ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:51:59.646518 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-ba8a1c29-1637-40b6-95e5-b6064206c3ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:51:59.655607 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a1c29-1637-40b6-95e5-b6064206c3ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:51:59.657060 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a1c29-1637-40b6-95e5-b6064206c3ea service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:51:59.665175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1297/2595] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:52:00.738686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:52:00.740368 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:52:00.754624 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e076c-bbc2-43e6-b404-172b16f9bbbf service nova] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:52:00.761081 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9647f8caa6cd4208800e2371fdf28301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:52:00.761704 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:52:00.761984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:52:00.762265 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:52:00.764577 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:52:00.776974 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:52:00.778574 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:52:00.789865 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1299/2596] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:52:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:52:02.307502 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a1c29-1637-40b6-95e5-b6064206c3ea service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:52:02.317092 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a1c29-1637-40b6-95e5-b6064206c3ea service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:52:02.320780 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536103f-9d6a-4fd8-9d69-06bcb3ac52ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4b023dbf114bf98d0e38ea7fec94fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:52:02.321379 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536103f-9d6a-4fd8-9d69-06bcb3ac52ec service neutron] REQUEST_METHOD: `GET` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:52:02.321657 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536103f-9d6a-4fd8-9d69-06bcb3ac52ec service neutron] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:52:02.321932 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536103f-9d6a-4fd8-9d69-06bcb3ac52ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:52:02.323685 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-7536103f-9d6a-4fd8-9d69-06bcb3ac52ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:52:02.376811 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=100 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:52:02.388698 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536103f-9d6a-4fd8-9d69-06bcb3ac52ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:52:02.390145 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536103f-9d6a-4fd8-9d69-06bcb3ac52ec service neutron] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:52:02.399872 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1298/2597] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:52:08.710595 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:52:08.712794 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:52:08.729788 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f98827-e210-43fd-b093-846cc694e238 service nova] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:52:08.735534 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:52:08.736268 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:52:08.736660 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:52:08.737002 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] PATH_INFO: `/v3/users/d7383d936b3544ccb9093a3beeb5cbd3` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:52:08.738196 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7383d936b3544ccb9093a3beeb5cbd3)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:52:08.740175 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:52:08.789201 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7383d936b3544ccb9093a3beeb5cbd3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:08.796031 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7383d936b3544ccb9093a3beeb5cbd3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:08.797632 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7383d936b3544ccb9093a3beeb5cbd3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:08.798247 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7383d936b3544ccb9093a3beeb5cbd3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:08.806234 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7383d936b3544ccb9093a3beeb5cbd3'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:08.825488 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1300/2598] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:52:08 2022] DELETE /identity/v3/users/d7383d936b3544ccb9093a3beeb5cbd3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:52:09.003783 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536103f-9d6a-4fd8-9d69-06bcb3ac52ec service neutron] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:52:09.013791 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536103f-9d6a-4fd8-9d69-06bcb3ac52ec service neutron] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:52:09.073864 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:52:09.074495 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:52:09.074776 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:52:09.075050 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] PATH_INFO: `/v3/projects/370ff2b623ce4dfc992a266bd41ee51e` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:52:09.076175 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=370ff2b623ce4dfc992a266bd41ee51e)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:52:09.078370 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:52:09.126982 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370ff2b623ce4dfc992a266bd41ee51e', 'request_id': 'req-816ea04a-1037-407f-85a4-b30fcfb05688'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.133686 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370ff2b623ce4dfc992a266bd41ee51e', 'request_id': 'req-816ea04a-1037-407f-85a4-b30fcfb05688'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.141830 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370ff2b623ce4dfc992a266bd41ee51e', 'request_id': 'req-816ea04a-1037-407f-85a4-b30fcfb05688'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.152782 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] The token cache is being invalidate because project 370ff2b623ce4dfc992a266bd41ee51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:52:09.153066 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.153681 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.155164 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1299/2599] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:52:09 2022] DELETE /identity/v3/projects/370ff2b623ce4dfc992a266bd41ee51e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:52:09.159984 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:52:09.161386 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:52:09.175236 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a24908-af87-4e08-bc57-74e519ac8ac2 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:52:09.215117 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:52:09.232318 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:52:09.232969 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:52:09.233333 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:52:09.233536 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] PATH_INFO: `/v3/users/00841c795d1b4f0e9515e8b725245cee` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:52:09.235470 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00841c795d1b4f0e9515e8b725245cee)` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:52:09.237415 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:52:09.281453 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00841c795d1b4f0e9515e8b725245cee'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.288271 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00841c795d1b4f0e9515e8b725245cee'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.289029 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00841c795d1b4f0e9515e8b725245cee'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.289634 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00841c795d1b4f0e9515e8b725245cee'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.297472 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.notifications [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00841c795d1b4f0e9515e8b725245cee'} {{(pid=82084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.314167 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1301/2600] 10.222.0.13 () {58 vars in 1224 bytes} [Thu Oct 20 14:52:09 2022] DELETE /identity/v3/users/00841c795d1b4f0e9515e8b725245cee => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:52:09.491287 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] Authenticating user token {{(pid=82085) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:52:09.492846 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.fernet_utils [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82085) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:52:09.509303 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ea04a-1037-407f-85a4-b30fcfb05688 admin admin] Validating token access rules against request {{(pid=82085) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:52:09.568200 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ee7ae2d2194fbf9ae79851bb412a22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa845f715934496950bc728f50bbff3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82085) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:52:09.568876 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:52:09.569157 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:52:09.569430 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] PATH_INFO: `/v3/projects/3c71e87c7020403982945a2019de5bc5` {{(pid=82085) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:52:09.570549 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c71e87c7020403982945a2019de5bc5)` {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:52:09.572471 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] RBAC: Authorization granted {{(pid=82085) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:52:09.597538 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:52:09.620119 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c71e87c7020403982945a2019de5bc5', 'request_id': 'req-efb90d90-ae0d-4061-a41d-ede846c00f31'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.627332 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c71e87c7020403982945a2019de5bc5', 'request_id': 'req-efb90d90-ae0d-4061-a41d-ede846c00f31'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.634448 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c71e87c7020403982945a2019de5bc5', 'request_id': 'req-efb90d90-ae0d-4061-a41d-ede846c00f31'} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.646054 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] The token cache is being invalidate because project 3c71e87c7020403982945a2019de5bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82085) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 14:52:09.646344 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.647010 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG keystone.notifications [None req-efb90d90-ae0d-4061-a41d-ede846c00f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82085) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 14:52:09.648545 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: [pid: 82085|app: 0|req: 1300/2601] 10.222.0.13 () {58 vars in 1233 bytes} [Thu Oct 20 14:52:09 2022] DELETE /identity/v3/projects/3c71e87c7020403982945a2019de5bc5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 14:52:19.310812 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:52:19.643151 dv-cinder-779233-7-iscsi devstack@keystone.service[82085]: DEBUG dbcounter [-] [82085] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=82085) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 14:54:09.862272 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] Authenticating user token {{(pid=82084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 14:54:09.863678 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:54:09.932878 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0dd82-a5db-4339-bff0-8e21de21dec0 admin admin] Validating token access rules against request {{(pid=82084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 14:54:09.990686 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e42837-13d6-4b99-9a02-592f06a5518f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc18a78229064e8d8ea4321c3c23c4ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e49ab75eb6fe499a901d76ddc3ff8027', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 14:54:09.991290 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42837-13d6-4b99-9a02-592f06a5518f service placement] REQUEST_METHOD: `GET` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 14:54:09.991566 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42837-13d6-4b99-9a02-592f06a5518f service placement] SCRIPT_NAME: `/identity` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 14:54:09.991837 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42837-13d6-4b99-9a02-592f06a5518f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 14:54:09.994644 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.fernet_utils [None req-23e42837-13d6-4b99-9a02-592f06a5518f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 14:54:10.006306 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e42837-13d6-4b99-9a02-592f06a5518f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 14:54:10.007862 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e42837-13d6-4b99-9a02-592f06a5518f service placement] RBAC: Authorization granted {{(pid=82084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 14:54:10.017965 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: [pid: 82084|app: 0|req: 1302/2602] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Oct 20 14:54:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 14:54:20.013390 dv-cinder-779233-7-iscsi devstack@keystone.service[82084]: DEBUG dbcounter [-] [82084] Writing DB stats keystone:SELECT=58 {{(pid=82084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}